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Autonomy  and  Networking  Technologies 


Individual  members  in  the  group  can 
®  sense  its  immediate  environment 

•  communicate  with  others 

®  process  the  information  gathered 

•  take  a  local  action  in  response 
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Prototypical  Dynamic  Vehicle  Routing  Problem 


Given: 

®  a  group  of  vehicles,  and 
®  a  set  of  service  demands 

Objective: 

provide  service  in  minimum  time 
service  =  take  a  picture  at  location 


Vehicle  routing  (All  info  known  ahead  of  time,  Dantzig  ’59) 


Determine  a  set  of  paths  that  allow  vehicles  to  service  the  demands 


Dynamic  vehicle  routing  (New  info  in  real  time,  Psaraftis  ’88) 


o  New  demands  arise  in  real-time 
9  Existing  demands  evolve  over  time 
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Light  and  heavy  load  regimes 
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Literature  on  DVR  and  queueing  for  robotic  networks 


9  Shortest  path  through  randomly-generated  and  worst-case  points 

(Beardwood,  Halton  and  Hammersly,  1959  —  Steele,  1990) 

9  Traveling  salesman  problem  solvers  (Lin,  Kernighan,  1973) 

9  DVR  formulation  on  a  graph  (Psaraftis,  1988) 

9  DVR  on  Euclidean  plane  (Bertsimas  and  Van  Ryzin,  1990-1993) 

9  Unified  receding-horizon  policy  (Papastavrou,  1996) 

Recent  developments  in  DVR  for  robotic  networks: 

9  Adaptation  and  decentralization 

9  Vehicles  with  dynamics,  nonholonomic  vehicles,  Dubins  UAVs 
9  Pickup  &  delivery  tasks 
9  Heterogeneous  vehicles  and  team  forming 
9  Distinct-priority  and  impatient  demands 
9  Moving  demands 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  1/8)  29junl0  @  Baltimore,  ACC  10  /  18 

Contributions  of  our  recent  works 


Comprehensive  framework  for  DVR  in  robotic  systems 

O  adaptive  DVR  policies  for  single  vehicles  in  light  and  heavy  load 

Q  cooperative  DVR  policies  via  partitioning 

Q  scalable  distributed  partitioning  policies  under  a  variety  of 
communication/interaction  scenarios 

Q  (models,  algorithms  and  analysis  of)  service  vehicles  with  dynamics 
&  stochastic  and  combinatorics  of  nonholonomic  Dubins  vehicles 
performing  Traveling  Salesman  Problems  and  DVR  tasks 

Q  (models,  algorithms  and  analysis  of)  service  vehicles  with  time 
constraints  and  heterogeneous  priorities 

Q  (models,  algorithms  and  analysis  of)  demands  requiring  service  by 
multiple  heterogeneous  vehicles  simultaneously. 
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Plain-vanilla  re-optimization? 


Example:  DVR  on  segment 


o  Objective:  minimize  average 
waiting  time 

o  Strategy:  re-optimize  at  each 
event 


For  adversarial  target  generation,  vehicle  travels  forever  without  ever 
servicing  any  request  => 

Even  if  queue  remains  bounded,  what  about  performance?  how  far 
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A  w  1  * 

o  Objective:  minimize  average 
waiting  time 

o  Strategy:  re-optimize  at  each 
event 

9  >  1  7 

0  0.5  1 

9  For  adversarial  target  generation,  vehicle  travels  forever  without  ever 
servicing  any  request  =>  unstable  queue  of  outstanding  requests 

9  Even  if  queue  remains  bounded,  what  about  performance?  how  far 
from  the  optimal? 
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Online  algorithms? 


Lecture  outline 


Online  algorithms  (Jaillet  and  M.  R.  Wagner  ’06) 


a  online  algorithm  operates  based  on  input  information  up  to  the 
current  time 

o  online  algorithm  is  (worst-case)  r-competitive  if 

Coston|ine(/)  <  rCostoptima|  offline ( / ) 5  V  problem  instances  /. 


Disadvantages 


Q  cumulative  cost 
Q  worst-case  analysis 

O  not  possible  to  include  a-priori  information  (e.g.,  arrival  rate) 
Q  not  as  clear  what  competitive  ratio  means 
Q  so  far,  only  few  simple  DVR  problems  admit  online  algorithms 
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Workshop  Structure  and  Schedule 


8:00-8:30am 

Coffee  Break 

8:30-9:00am 

Lecture  #1: 

9:05-9:50am 

Lecture  #2: 

9:55-10:40am 

Lecture  #3: 

10:40-ll:00am 

Break 

ll:00-ll:45pm 

Lecture  #4: 

ll:45-l:10pm 

Lunch  Break 

l:10-2:10pm 

Lecture  #5: 

2:15-3:00pm 
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3:00-3:20pm 
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3:20-4:20pm 

Lecture  #7: 

4:25-4:40pm 

4:45-5:00pm 

Lecture  #8: 

Intro  to  dynamic  vehicle  routing 
Prelims:  graphs,  TSPs  and  queues 
The  single-vehicle  DVR  problem 

The  multi-vehicle  DVR  problem 

Extensions  to  vehicle  networks 
Extensions  to  different  demand  models 

Extensions  to  different  vehicle  models 
Extensions  to  different  task  models 
Final  open-floor  discussion 
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Key  references  for  this  lecture 


Graph  Theory  Basics: 

R.  Diestel.  Graph  Theory ,  volume  173  of  Graduate  Texts  in  Mathematics.  Springer, 
2  edition,  2000 

Combinatorial  Optimization: 

B.  Korte  and  J.  Vygen.  Combinatorial  Optimization:  Theory  and  Algorithms,  vol¬ 
ume  21  of  Algorithmics  and  Combinatorics.  Springer,  4  edition,  2007 

Stochastic  TSP: 

J.  M.  Steele.  Probability  Theory  and  Combinatorial  Optimization.  SIAM,  1987 

Basic  Queueing  Theory: 

L.  Kleinrock.  Queueing  Systems.  Volume  I:  Theory.  Wiley,  1975 
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Q  Queueing  Theory 

o  Kendall’s  Notation 
o  Little’s  Law  and  Load  Factor 
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Graph  Theory  Review 


o  An  undirected  graph  G  =  (V,E). 

9  a  path  in  G  is  a  sequence  vi,  ei,  . . . ,  v/c,  e^,  v^+i,  with 
®  e#-  7^  ej  for  /  7^7. 
o  Vj  7^  Vj  for  all  /  7^  j. 

o  A  circuit  or  cycle  has  vi  =  v/c+i. 

o  A  lamiltonian  pat  is  a  path  that  contains  all  vertices, 
o  Similarly  define  a  Hamiltonian  cycle  or  tour. 
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Minimum  Spanning  Tree 


®  A  tree  is  a  graph  with  no  cycles 

9  A  spanning  tree  of  G  is  a  subgraph 
that 

Q  is  a  tree 

o  connects  all  vertices  together 


Can  be  solved  in  greedy  fashion  using  Kruskal’s  algorithm: 

9  Recursively  adds  shortest  edge  that  does  not  create  a  cycle 
9  Runs  in  0(n2)  time  (where  \  V\  —  n) 
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Weighted  Graphs 


9  A  weighted  graph  G  —  (V,  E,c)  has  edge  weights  c  :  E  — ►  M>o- 

9  In  a  complete  graph,  E  —  V  x  V. 

Special  classes  of  complete  weighted  graphs: 

9  Metric  if 

c({vi,  v2})  +  c({v2,  v3})  >  c({vi,  v3})  for  all  vu  v2,  v3  e  V. 

9  Euclidean  if 

V  C  M.d  and  c({v,-,  vj})  =  || v;  —  vj\\2. 
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Hamiltonian  Cycle  Decision  Problem 


Hamiltonian  Cycle  is  complete 

(One  of  Karp’s  21  NP-complete  problems) 

Recall,  a  problem  is  NP-complete  if 

o  Every  solution  can  be  verified  in  polynomial  time  (NP). 

9  Every  problem  in  NP  can  be  reduced  to  it. 
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Traveling  Salesman  Problem 


®  TSP  is  NP-hard 

9  To  show  NP-hard:  Reduce  Hamiltonian  Cycle  to  TSP. 

Given  an  undirected  graph  G  =  (V,E)  with  |  V|  =  n: 

9  Construct  complete  graph  Gn  with  weight  1  for  each  edge  in  E  and 
weight  2  for  all  other  edges. 

Q  Then  G  is  Hamiltonian  o  optimum  TSP  tour  has  length  n. 
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Outline 
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Metric  TSP 


®  The  Metric  TSP  is  l\IP-hard. 

®  There  exist  approximation  algorithms! 
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Double-Tree  Algorithm 


Double-Tree  Algorithm 

1:  Find  a  minimum  spanning  tree  T  of  graph  Gn. 

2:  G  :=  graph  containing  two  copies  of  each  edge  in  7”. 
3:  Compute  Euler ian  walk  in  Euler ian  graph  G. 

4:  Walk  gives  ordering,  ignore  all  but  first  occurrence 
of  vertex. 
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Eulerian  Graphs 


o  Eulerian  graph:  degree  of  each  vertex  is  even 
o  Eulerian  walk  Closed  walk  containing  every  edge. 

<*  Graph  has  Eulerian  walk  <^>  Eulerian. 

9  Eulerian  walk  can  be  computed  in  0(|\/|  +  \E\)  time. 
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Christofides’  Algorithm 


Christofides'  Algorithm 

1:  Find  a  minimum  spanning  tree  T  of  G . 

2:  Let  1/1/  be  the  set  of  vertices  with  odd  degree  in  T. 

3:  Find  the  minimum  weight  perfect  matching  M  in 
subgraph  generated  by  1/1/. 

4:  Find  an  Eulerian  path  in  G  :=  (\/n,E(7”)  U  M) ,  (skip 

vertices  already  seen) . 
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Double-Tree  Algorithm 


Theorem 


Double-Tree  Algorithm  is  a  2-approx  algorithm  for  the  Metric  TSP.  Its 
running  time  is  0(n2). 


9  Deleting  one  edge  from  a  tour  gives  a  spanning  tree. 

9  Thus  minimum  spanning  tree  is  shorter  than  optimal  tour. 
q  Each  edge  is  doubled. 

9  Spanning  tree  can  be  computed  in  0(n2)  time. 

9  Eulerian  walk  computed  in  0(n)  time. 
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Christofides’  Algorithm 


Theorem 


Christofides’  Algorithm  gives  a  3/2 -approx  algorithm  for  the  Metric  TSP. 
Its  running  time  is  0(n3). 


®  /.(Christofides)  =  /.(MST)  +  L(M). 

*  But,  /.(MST)  <  /.(TSP),  and 
®  L(M)  <  L(M')  <  L( TSP)/2. 

Where  M'  is  the  minimum  perfect  matching  of  1/1/  using  edges  that 
are  part  of  TSP. 


Best  known  approx  algorithm  for  Metric  TSP 
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Length  Bounds  for  Euclidean  TSP 


How  long  is  the  TSP  tour  through  n  points  in  unit  square? 


Worst-case  lower  bound  matches: 

®  Equally  space  n  points  on  a  grid 
®  Then  ETSP(Q„)  =  C^fh. 

®  So,  worst-case  length  >  C^fn. 
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Euclidean  TSP 


Practical  value  limited  to  due  c’s  dependence  on  e. 
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Worst-case  TSP  Length  Upper  Bound  (Intuition) 


®  Consider  Qn  {xi, . . .  ,x„}  of  n  points  in  unit  square. 
®  There  exists  c  >  0  such  that 

min  {||x(-  -  xj\\  :  x,-,xy  e  (?„} 

®  Let  £n  denote  worst-case  TSP  length  through 
®  Then  £n  <  £n~i  +  2 c/yT. 

®  Summing  we  get  £(n)  <  C^n. 


n  pts. 
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Worst-case  TSP  Length  Upper  Bound  (Intuition) 
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\  n 
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TSP  Length  for  Random  Points 


Theorem  (Beardwood,  Halton,  and  Hammersley,  1959) 


Let  Qn  be  a  set  of  n  i.i.d.  random  variables  with  compact  support  in 
and  distribution  p(x).  Then,  with  prob.  1 


ETSP  (<?„) 

lim  — 7~7 
n— >+oo 


—  0TSP,d  /  V(x)id  1)/ddx, 


where  /?jsp ,d  /s  3  constant  independent  of  ip,  and  (p  is  absolutely 
continuous  part  of  ip. 


For  uniform  distribution  in  square  of  area  A 

ETSP(a)  „  m 

- -j= - Ptsp ,2v  A  as  n  — ►  +oo. 

Vn 

Best  estimate  of  /?tsp,2  is  Percus  and  Martin,  1996 

/3tsp,2  —  0.7120. 
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Outline 


Q  Queueing  Theory 

9  Kendall’s  Notation 
o  Little’s  Law  and  Load  Factor 
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Summary  of  Traveling  Salesman  Problem 


a  Solving  TSP  is  -har  ,  and  no  approx  algorithms  exist, 
o  For  metric  1  ,  still  /VP-hard  but  good  approx  algs  exist. 

9  For  Euclidean  TSP,  very  good  heuristics  exist. 

9  Length  of  tour  through  n  points  in  unit  square: 

o  Worst-case  is  Q(y/n). 

9  Uniform  random  is  Q(yfn). 

9  For  all  density  functions  0(^/n). 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  2/8)  29junl0  ®  Baltimore,  ACC  22  /  29 


Basic  Queueing  Model 


9  Customers  arrive,  wait  in  a  queue,  and  are  then  processed 
9  Queue  length  builds  up  when  arrival  rate  is  larger  than  service  rate 


queue  length 

9  Arrivals  modeled  as  stochastic  process  with  rate  A 
9  Service  time  of  each  customer  is  a  r.v.  with  finite  mean  s  and 
second  moment  s2. 

9  Service  rate  is  1/s. 
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Queueing  Notation 


Kendall’s  Queueing  notation  A/B/C: 

o  A  —  the  arrival  process 
o  B  =  the  service  time  distribution 
o  C  =  the  number  of  servers 


Main  codes: 

9  D  =  Deterministic 
o  M  =  Markovian 

o  for  arrivals:  Poisson  process 
o  for  service  times:  Exponential  distribution 

o  G  (or  G/)  =  General  distribution  (independent  among  customers) 

Example  M/G/m  queue: 

o  Poisson  arrivals  with  rate  A 
9  General  service  times  with  mean  s 
o  m  servers 
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Wait-time  examples 


For  M/D/1  queue: 


For  M/G/l  queue: 


W= 


2(1-0) 


w=  As2 


2(1  ~e) 


For  G/G/l  queue  (Kingman,  1962): 

-  2(1  -e) 

and  the  upper  bound  becomes  exact  as  g  — >  1“ 
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Little’s  Law  and  Load  Factor 


Define: 

®  average  wait-time  in  queue  as  W 
®  average  system  as  T  :=  1/1/  +  s. 


®  For  m  servers,  define  load  factor  as 

As 

Q  :=  — 

m 

®  Necessary  condition  for  stable  queue  is  Q  <  1- 
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The  problem 


1 


DVR  -  distinct  features 


»  service  demands  vary  over  time 

9  information  about  future  is 
stochastic 

9  real-time  routing  policies 
9  queueing  phenomena 


DVR  is  fundamentally  a  queueing  problem: 
Q  arrival  process 
Q  service  model 
Q  performance  measure 
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Lecture  outline 


Q  Queueing-theoretical  model  for  DVR 


D.  J.  Bertsimas  and  G.  J.  van  Ryzin.  A  stochastic  and  dynamic  vehicle  routing  problem  in  the 
Euclidean  plane.  Operations  Research,  39:601-615,  1991 
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General  queueing-theoretical  model  for  DVR  1/2 


Arrival  process:  spatio-temporal  Poisson 
O  time  intensity  A  >  0 

Q  spatial  density  <p\  P  [demand  in  <S]  =  Js  </>(x)  dx 
Q  inter-arrival  times  and  locations  are  i.i.d. 
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General  queueing-theoretical  model  for  DVR  1/2 


Arrival  process:  spatio-temporal  Poisson 
G  time  intensity  A  >  0 

G  spatial  density  ip\  P  [demand  in  <S]  =  Js  </>(x)  dx 
G  inter-arrival  times  and  locations  are  i.i.d. 

Service  model: 

G  m  holonomic  vehicles  with  maximum  velocity  v 
G  vehicles  provide  a  random  on-site  service 
G  on-site  service  times  are  i.i.d.  (equal  on  average  to  s) 

G  demand  removed  from  the  system  upon  on-site  service  completion 
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Relation  to  standard  queueing  systems 


9  DVR  model  close  to  M/G/m  queue 
9  key  difference:  service  times  are  lot  i.i.d.  in  general 


Service  time  correlations  in  DVR: 

9  service  time  =  travel  time  +  on-site  service 
9  FCFS  policy 

9  unconditional  expected  travel  time 

between  two  consecutive  demands  ~  0.52. 
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General  queueing-theoretical  model  for  DVR  2/2 


Performance  measure:  steady-state  system  time  of  demands  T 
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Relation  to  standard  queueing  systems 


9  DVR  model  close  to  M/G/m  queue 
9  key  difference:  service  times  are  lot  i.i.d.  in  general 


Service  time  correlations  in  DVR: 

9  service  time  =  travel  time  +  on-site  service 
9  FCFS  policy 

9  unconditional  expected  travel  time 

between  two  consecutive  demands  «  0.52. 

9  conditional  expected  travel  time  between 
two  consecutive  demands  >  0.52. 


M/G/m  methodology  is  not  applicable! 
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A  first  look  at  the  problem:  stability 


®  A  •  E[service  time]/m  fraction  of  time  each  vehicle  is  busy 


Necessary  condition  for  stability: 

System  is  stable  if  A  •  E[service  time]/m  <  1. 

Since  s  <  E[service  time],  a  weaker  necessary  condition  is: 

g  —  Xs/m  <  1 

Sufficient  condition  for  stability: 

Surprisingly,  g  <  1  is  also  sufficient  for  stability  =>  stability  condition 
independent  of  the  size  and  shape  of  Q 
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Lecture  outline 


Q  Lower  bounds  on  performance  (m=l) 


D.  J.  Bertsimas  and  G.  J.  van  Ryzin.  A  stochastic  and  dynamic  vehicle  routing  problem  in  the 
Euclidean  plane.  Operations  Research,  39:601-615,  1991 

D.  J.  Bertsimas  and  G.  J.  van  Ryzin.  Stochastic  and  dynamic  vehicle  routing  with  general  interar¬ 
rival  and  service  time  distributions.  Advances  in  Applied  Probability,  25:947-978,  1993 
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Analysis  approach 


o  Lack  of  i.i.d.  property  substantially  complicates  analysis 
9  General  approach: 

Q  lower  bounds  on  performance,  independent  of  algorithms, 

Q  design  of  algorithms  and  upper  bound  on  their  performance,  possibly  in 
asymptotic  regimes  (i.e.,  g  — »  0+  and  g  — »  1“) 
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Light-load  lower  bound 


Lower  bound  (most  useful  when  A  — >  0+) 


For  all  policies  7 r:  Tn  >  E^fHX  —  p* ||]/ 1/  +  s 

Proof  sketch: 

®  T  =  1/1/ travel  +  ^on-site  +  S 

®  wtrave,  >  E*[||X  -  P*\\]/v 
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Light-load  lower  bound 


Lower  bound  (most  useful  when  A  — >  0+) 


For  all  policies  7 r:  Tn  >  E^fHX  —  p* ||]/ 1/  +  s 

Proof  sketch: 

®  7"  =  H/travel  ~F  l^on-site  “F  S 

®  wtrave,  >  E„[||X  -  p*\\]/v 
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Light-load  lower  bound 


Proof  sketch: 

®  T  =  1/1/ travel  +  Won-site  +  S 

®  Wtrave,  >  E*[||X  -  p*\\]/v 
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Light-load  lower  bound 


Proof  sketch: 

®  7"  =  H/travel  “F  l^on-site  “F  S. 

®  Wtrave,  >  E„[||X  -  p*\\]/v 
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Heavy-load  lower  bound 


Definition  (Spatially-biased  and  -unbiased  policies) 


A  policy  7 r  is  said  to  be 

Q  spatially  unbiased  if  system  time  is  independent  of  demand  location 
Q  spatially  biased  if  system  time  depends  on  demand  location 
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Proof  sketch  (for  unbiased  policies) 


Proof  of  the  lower  bound: 

®  the  idea  is  to  use  stability  arguments  (which  are  independent  of 
policies!) 

o  let  D  be  the  travel  inter-demand  distance 
®  one  can  show  that 


D  >  /?TSP 


/Q¥>1/2(*)dx 

V2N 


as  g 


1", 


with  N  average  number  of  waiting  demands 
9  for  stability: 


D  1 

s  H - <  — 

v  A 


_  ,  „  Jq¥>1/2 (x)dx 

s  +  /?TSP  - 7^= -  <  1/A 


v  V2N 

9  since  N  —  X 1/1/  and  T  —  W  +  s  one  obtains: 

Ptsp  X  (/q^1/2Wc/x)2 


T  > 


V2  (1  -  q)2 
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Heavy-load  lower  bound 


0  Control  policies 


D.  J.  Bertsimas  and  G.  J.  van  Ryzin.  A  stochastic  and  dynamic  vehicle  routing  problem  in  the 
Euclidean  plane.  Operations  Research,  39:601-615,  1991 

D.  J.  Bertsimas  and  G.  J.  van  Ryzin.  Stochastic  and  dynamic  vehicle  routing  with  general  interar¬ 
rival  and  service  time  distributions.  Advances  in  Applied  Probability,  25:947-978,  1993 

M.  Pavone,  E.  Frazzoli,  and  F.  Bu No.  Distributed  and  adaptive  algorithms  for  vehicle  routing 
in  a  stochastic  and  dynamic  environment.  IEEE  Transactions  on  Automatic  Control,  May  2010. 
(Submitted,  Apr  2009)  to  appear 
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An  optimal  light  load  policy 


Stochastic  Queueing  Median  (SQM) 

Compute  median  p* .  Then: 

1:  service  demands  in  FCFS  order 
2:  return  to  p*  after  each  service  is 
completed 
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An  optimal  spatially-unbiased  heavy-load  policy 


Unbiased  TSP  (UTSP) 

Partition  Q  into  r  subregions  Q ^  with  Jq  ( p(x)dx 
Then: 

1:  within  each  subregion  form  sets  of  size  n/r 
2: 

3: 


=  1  /r- 


deposit  sets  in  a  queue 

service  sets  FCFS  by  following  a  TSP  tour 
Optimize  over  n. 


Optimality  of  UTSP  policy 


lim  TutspCO/Tu  <1  +  1  /r 

Q^l~ 
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An  optimal  light  load  policy 


Stochastic  Queueing  Median  (SQM) 

Compute  median  p* .  Then: 

1:  service  demands  in  FCFS  order 
2:  return  to  p*  after  each  service  is 
completed 


Optimality  of  SQM  policy 


lim  Tsqm/T  =  1 

A^0+ 


Proof  sketch 

o  As  A  ->  0+,  P  [demand  generated  when  system  is  empty] 

o  =>►  all  demands  generated  with  the  vehicle  at  p* 


Tsqm  =  Ey,[||X  -  p 


'  V  +  s 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  3/8)  29junl0  ©  Baltimore,  ACC  14  /  25 


Proof 


Proof  (r  =  1) 

o  idea:  reduction  to  G  queue 

9  j th  set  viewed  as  yth  customer:  arrival  and  service  times  are  .i.d  ! 
o  inter-arrival  distribution  is  Erlang  of  order  n 
©  expected  service  time  is  ns  +  (3 tsp  Jq(P1^2{x)(^x/v 

o  standard  results  give  upper  bound  on  the  wait  in  queue  for  a  set 
o  then  easy  to  find  upper  bound  for  individual  demands 
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Comments 


Relation  with  non-spatial  queueing  systems: 

®  wait  time  grows  as  (1  —  g)~2  instead  of  (1  —  £>)-1! 

®  DVR  problems  are  fundamentally  different  from  traditional  queueing 
systems  (techniques,  results,  etc.) 

Analysis  techniques: 

®  for  light  load:  locational  optimization 

®  for  heavy  load:  reduction  to  classic  queueing  systems  or 
control-theoretical  methods 

Biased/unbiased: 

®  biased  service  provides  strict  reduction  of  optimal  system  time  for  any 
non-uniform  tp 
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Divide  &  Conquer  policy 


Divide  &  Conquer  (DC) 

Partition  Q  into  r  subregions 
Qk  with  fQk  ip{x)dx  =  1/r. 

Then: 

1:  while  no  customers,  move 
to  empirical  median  p* 

2:  while  customers  waiting 

Q  move  to  subregion  Qk 

Q  service  all  demands  in 
Qk  by  following  a  TSP 
tour 

Q  k  < —  /c  — 1  (modulo  r) 
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Adaptivity 


SQM  policy  not  adaptive: 

®  SQM  unstable  as  g  — ►  1“ 

®  intuition:  average  per-demand  travel  D  is  :ixed 
®  but  stability  condition  implies  D  <  (1  -  g)/\\ 

UTSP  and  BTSP  policies  not  adaptive: 

®  for  stability  of  the  queue  of  sets: 


+  /?tsp  Vn  <p1/2(x)dx/ v'j 


<  1 


®  then  one  should  a  prio  select: 


n  >  A2/?^gp 


Uq 


V?1/2(x)  dx 


/(v2(l  -g)2) 


®  =>  wrong  selection  of  n  might  lead  to  instability  or  unacceptable 
deterioration  in  performance 
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DC  policy  (with  r  — >  +oo) 


Implementation: 

o  NP-hard  computation,  but  effective  heuristics 
Adaptation:  the  policy  does  not  require  knowledge  of 
Q  vehicle  velocity  v,  environment  Q 
Q  arrival  rate  A 
O  expected  on-site  service  s 

Performance: 

O  in  light  load,  delay  is  optimal 
O  in  heavy  load,  delay  is  optimal 
Q  stable  in  any  load  condition 


optimal  and  adaptive 
very  little  known  outside  of  asymptotic  regimes 
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Proof  (r=l) 


Light  load: 

®  p*  — >  p*  and  recovers  SQM 

Heavy  load: 

o  no  well-defined  notion  of  “_/th  customer" 

9  focus  on  dynamical  system 

r  n;  i 

E[n/+1]<AE  ^Sq  +  TSPCn/) 

q=l 

<  A  (s  E[n/]  +  /3Tsp  f  <p1/2(x)cfx  y/E[rii]/ v 

J  Q 


9  upper  bound  trajectories  with  the  trajectories  of  virtual  dynamical 
system 

Qz;  +  (\/v)/3tsp  [  1/2(x)dx 

Jo. 


Zi+l  = 

9  Tdc<  I  i  m  / — ».-|-oo  Zj/X 


lx  x/Z, 
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RH  policy 


Implementation: 

9  NP-hard  computation,  but  effective  heuristics 
Adaptation:  the  policy  does  not  require  knowledge  of 
Q  vehicle  velocity  v,  environment  Q 
Q  arrival  rate  A  and  spatial  density  function  cp 
O  expected  on-site  service  s 


Performance: 

O  in  light  load,  delay  is  optimal 

Q  in  heavy  load,  delay  is  within  a  multiplicative  factor  from  optimal 
Q  multiplicative  factor  depends  upon  <p  and  is  conjectured  to  equal  2 


adaptive  to  all  parameters 
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Receding-Horizon  policy 


Receding-Horizon  (RH) 

For  77  E  (0,1],  single  agent  performs: 

1:  while  no  customers,  move  to  empirical  median  p* 
2:  while  customers  waiting 

Q  compute  TSP  tour  through  current  demands 
Q  service  p-fraction  of  path 
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Lecture  outline 


Q  Queueing-theoretical  model  for  DVR 


Q  Lower  bounds  on  performance  (m= 1) 


Q  Control  policies 
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Lecture  outline 


Q  Territory  Partitioning 
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Load  balancing  in  DVR  via  territory  partitioning 


Q  Resource  allocation  in  DVR  is  transcribed  into  partitioning! 

Q  Focus  of  this  lecture  is  mutivehicle  DVR  via  optimal  partitioning 
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Territory  partitioning  is  ...  art 


Ocean  Park  Paintings,  by  Richard  Diebenkorn  (1922-1993) 
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Territory  partitioning:  optimality  and  behaviors 


DESIGN  of  performance  metrics 

Q  how  to  cover  a  region  with  n  minimum-radius  overlapping  disks? 

Q  how  to  design  a  minimum-distortion  (fixed-rate)  vector  quantizer? 
Q  where  to  place  mailboxes  in  a  city  /  cache  servers  on  the  internet? 


ANALYSIS  of  cooperative  distributed  behaviors 


how  do  animals  share  territory? 
how  do  they  decide  foraging 
ranges? 

how  do  they  decide  nest  locations? 

O  what  if  each  robot  goes  to  “center”  of  own  dominance  region? 
0  what  if  each  robot  moves  away  from  closest  vehicle? 
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Optimal  partitioning 


The  Voronoi  partition  {Vi, ,  Vn}  generated  by  points  (pi, . . . ,  pn ) 

V(p)  =  {x  6  Q\  ||x  -  p/||  <  ||x  -  p/ll ,  Y j  V  /'} 

=  QP|(half  plane  between  /  and  j,  containing  /) 


Descartes  1644,  Dirichlet  1850,  Voronoi  1908,  Thiessen  1911, 
Fortune  1986  (sweepline  algorithm  0(n\og(n))) 
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Multi-center  functions 


V)  =  J2  f(llx  “  Pi\\Mx)dx 

i= i  Jv> 

o  <p  :  M2  — >•  ]R>o  density 
®  f  :  M>o  — >  R  penalty  function 

F.  Bullo,  J.  Cortes,  and  S.  Martinez.  Distributed  Control  of  Robotic  Networks.  Applied  Mathematics 
Series.  Princeton  University  Press,  2009.  Available  at  http://www.coordinationbook.info 
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Optimal  centering  (for  region 

v  with  density  </?) 

function  of  p 

minimizer  =  center 

llx  -  P\\<fi(x)dx 

J  V 

median  (or  Fermat-Weber  point) 

||x  -  p||2<p(x)dx 

J  V 

troid  (or  cen  mass) 

p  i— >  area(v  n  disk(p,  r)) 

r- area  center 

p  i— >  radius  of  largest  disk  centered 

incenter 

at  p  enclosed  inside  v 

p  i— >  radius  of  smallest  disk  cen¬ 

circumcenter 

tered  at  p  enclosing  v 

From  online 
Encyclopedia  of 
Triangle  Centers 
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How  to  compute  the  median  of  a  convex  set 


For  convex  planar  set  Q  with  strictly  positive  density  ip, 


^fw(p)  =  [ 

Jq 


p-x  W<p(x)dx 


O  Wfw  is  strictly  convex 

O  the  global  minimum  point  is  in  Q  and  is  called  median  of  Q 
O  compute  median  via  gradient  flow  with 

^t^fw(p)  =  f  7T- — TrMx)dx 
dp  Jq  Wp  ~  x|| 
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Gradient  algorithm  for  multicenter  function 


After  assuming  v  is  Voronoi  partition, 

n(p)  =  J2[  f(\\x  -  Pj\\Mx)dx 

~i  JVj(p) 

For  f  smooth,  note  simplifications  for  boundary  terms 

£f(p)=/  -S-f(.\\x-Pi\\)<p(x)dx 

°Pi  JVi(p)  vPi 
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From  optimality  conditions  to  algorithms 


n(Piv)  =  J2  [  f(\\x~  P/HM*)d* 

/=!  Jv' 


Theorem  (Alternating  Algorithm,  Lloyd  ’57) 


O  at  fixed  positions,  optimal  partition  is  Voronoi 
Q  at  fixed  partition,  optimal  positions  are  ' generalized  centers " 

Q  alternate  v-p  optimization 

= =>  local  optimum  =  center  Voronoi  partition 
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Gradient  algorithm  for  multicenter  function 


After  assuming  v  is  Voronoi  partition, 

n(p)  =  J2[  f(\\x  -  Pj\\Mx)dx 

JVj(p) 

For  f  smooth,  note  simplifications  for  boundary  terms 


&H 

dp, 


~(P)=  f  -§-f{\\x~  Pi\\)v{x)dx 

i  JVi{p)  d Pi 

+  /  f(\\x-Pi\\){ni(x),j£)<P(x)dx 

JdVi(p)  dpi 
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Gradient  algorithm  for  multicenter  function 


After  assuming  u  is  Voronoi  partition, 

n(p)  =  Yl  \  f(\\x  -  Pj\\Mx)dx 

Jvj(p) 

For  f  smooth,  note  simplifications  for  boundary  terms 

?^(p)  =  [  ^-HWx-PilDvWdx 

OPi  JVj(p)  OPi 

+  [  f(.\\x-Pi\\){ni(x),^-)<p(x)dx 

JdVi(p)  OPi 

+  [  H\\x-  Pj\\)  (nji{x),  ^-)ip{x)dx 

j  ntigh  /  JdVj(p)ndVi(p)  dp, 

V - V - ' 

contrib  from  neighbors 
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Example  optimal  partition 
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Gradient  algorithm  for  multicenter  function 


After  assuming  v  is  Voronoi  partition, 


H(p)  =  J2[  f(\\x  -  Pj\\)v(x)dx 

7 =x  Jvj(p) 


For  f  smooth,  note  simplifications  for  boundary  terms 

£f(p)=/  w-f(\\x-Pi\\)<p{x)dx 

°Pi  JVi(p)  dPi 


[  f(\\x-Pi\\){n,(x),^-)(p(x)dx 

'dVi(p)  dp, 

[  f{\\x-Pi\\){n,(x),  ^-)(p{x)dx 

IdVj(P)  OPi 
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Lecture  outline 


^  The  multi-vehicle  DVR  problem 


D.  J.  Bertsimas  and  G.  J.  van  Ryzin.  Stochastic  and  dynamic  vehicle  routing  with  general  interar¬ 
rival  and  service  time  distributions.  Advances  in  Applied  Probability,  25:947-978,  1993 
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Multi-vehicle  DVR  problem 


o  results  on  single-vehicle  DVR  generalize  easily  to  the  multi-vehicle 
case 

9  previous  methodology  (locational  optimization,  queueing  and  control 
theory,  combinatorics)  applicable  to  this  case 

o  main  new  idea:  partitioning 
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Heavy-load  lower  bound 


r  1 

Heavy-load  lower  bound 

B2  A  1 

spatially-unbiased  policies:  Tn  >  s  — 

B2  A  1 

-  M'T'QTD 

spatially-biased  policies:  Tn  >  ^  — 

m2  v2  (1  —  q)2 

(/q  ^2/3(x)dx) 

m2  v2  (1  —  q)2 

2 

1 

—  as  q  — >  1 

3 

1 

—  as  q  — ►  1 
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Light-load  lower  bound 


Proof  sketch: 


o  multi-median:  best  a  priori  location 
to  reach  a  newly  arrived  demand 
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An  optimal  light-load  policy 


m  Stochastic  Queueing  Median 
(mSQM) 

Compute  multi-median  p*  and 
assign  one  vehicle  at  each 
median  point.  Then: 

1:  Assign  demand  that  falls 
in  Vi  to  vehicle  / 

2:  each  vehicles  service 
demands  in  FCFS  order 
3:  each  vehicle  returns  to 
pk  after  each  service  is 
completed 
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An  optimal  spatially-unbiased  heavy-load  policy 


Unbiased  TSP  (UTSP) 

Partition  Q  into  r  subregions 

Qk  with  fQk 

(p(x)dx  =  1/r. 

Then: 

1:  within  each  subregion  form 

sets  of  size 

i  n/r 

2:  deposit  sets  in  a  queue 

3:  service  sets  FCFS  with  the 

first  available  vehicle  by 

following  a  TSP  tour 

Optimize  over  n. 
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An  optimal  light-load  policy 


m  Stochastic  Queueing  Median 

(mSQM) 

Compute  multi-median  p*  and 

assign  one  vehicle  at  each 

A 

V8 

• 

median  point.  Then: 

• 

• 

1:  Assign  demand  that  falls 

• 

• 

in  Vi  to  vehicle  / 

o 

• 

2:  each  vehicles  service 

■* 

• 

demands  in  FCFS  order 

«  • 

@ 

3:  each  vehicle  returns  to 

a 

p*k  after  each  service  is 

completed 

Proof  sketch  of  optimality 

o  As  A  — >  0+ ,  P  [demand  generated  when  system  is  empty]  — >  1 

»  =>  all  demands  are  generated  with  the  vehicles  at  p* 

FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  4/8) 

29junl0  ©  Baltimore,  ACC  17  /  27 
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Lecture  outline 


^  Multi-vehicle  DVR  policies  based  on  partitioning 


M.  Pavone,  E.  Frazzoli,  and  F.  Bullo.  Distributed  and  adaptive  algorithms  for  vehicle  routing 
in  a  stochastic  and  dynamic  environment.  IEEE  Transactions  on  Automatic  Control,  May  2010. 
(Submitted,  Apr  2009)  to  appear 
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Motivation 


Performance: 

®  light  load:  problem  reduces  to  locational  optimization 
®  heavy  load: 

Q  delay  of  optimal  single  vehicle  policy  scales  as  A  \  Q\ 
Q  by  (equitably)  partitioning,  delay  reduces  to  ^  ^ 

Q  =>•  delay  scales  as  m~2,  as  in  the  lower  bound 
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Motivation 


Performance: 

o  light  load:  problem  reduces  to  locational  optimization 
a  heavy  load: 

Q  delay  of  optimal  single  vehicle  policy  scales  as  A  \Q\ 

Q  by  (equitably)  partitioning,  delay  reduces  to  ^  ^ 

O  =>  delay  scales  as  m~ 2,  as  in  the  lower  bound 

Implementation: 

a  systematic  approach  to  lift  adaptive  single-vehicle  policies  to 
multi-vehicle  policies 

o  coupled  with  listributec  partitioning  algorithms,  provides  distributed 
multi-vehicle  policies 
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Motivation 


Performance: 

9  light  load:  problem  reduces  to  locational  optimization 
9  heavy  load: 

Q  delay  of  optimal  single  vehicle  policy  scales  as  A  \Q\ 

Q  by  (equitably)  partitioning,  delay  reduces  to  ^  ^ 

O  =>  delay  scales  as  m~2,  as  in  the  lower  bound 

Implementation: 

*  systematic  approach  to  lift  adaptive  single-vehicle  policies  to 
multi-vehicle  policies 

9  coupled  with  listributec  partitioning  algorithms,  provides  distributed 
multi-vehicle  policies 

distributed  multi-vehicle  policy  =  single-vehicle  policy  +  optimal 
partitioning  +  distributed  algorithm  for  partitioning 
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Optimal  partitioning  in  heavy  load 


Intuition 

®  per-vehicle  workload  is  oc  A  fg (f(x)dx 

®  per-vehicle  service  capacity  is  oc  A  /q*  V1,2(x)dx 
9  optimal  partitioning  =  equalizing  per-vehicle  workload  am  service 
capacity 
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Optimal  partitioning  in  heavy  load 


Intuition 

®  per-vehicle  workload  is  oc  A  fg  ip(x)dx 
®  per-vehicle  service  capacity  is  oc  A  /q* 

9  optimal  partitioning  =  equalizing  per-vehicle  workload  am  service 
capacity 
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Optimal  partitioning  in  heavy  load 


Intuition 

®  per-vehicle  workload  is  oc  A  Jq  ip(x)dx 

®  per-vehicle  service  capacity  is  oc  A  /q*  V1/2(x)dx 
9  optimal  partitioning  =  equalizing  per-vehicle  workload  am  service 
capacity 


Simultaneously  equitable  partitions  exist  for  any  Q  and  <p 

(S.  Bespamyatnikh,  D.  Kirkpatrick,  and  J.  Snoeyink,  2000) 
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Optimal  partitioning  in  heavy  load 


Theorem 


Given  single-vehicle  optimal  policy  ir*,  a  tt*- partitioning  policy  using  a 
simultaneously  equitable  partition  is  an  optimal  unbiased  policy 


Proof  sketch 

»  P  [demand  arrives  in  Qk]  —  fg  </?(x)  dx  —  1/m 

9  arrival  rate  in  region  k:  Xk  —  A/m 
9  =>  Qk  =  XkS  =  As/ m  =  g  <  1  =y  system  is  stable 


9  conditional  density  for  region  k\  (P(x)/(fgk  </7(x)  dx'j  —  mip(x) 

•  T  =  HU  (  /&  f(«)  ^ 

Em  T  ~r  _T_ 

k=l  m  1  m2 


fc 


Qk  \l  fak  ^  dx 


dx 
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Comments 


If  {Qk}k=i  's  only  equitable  wrt  to  ip1/2... 

9  3k  such  that  —  X  [1  / m  -\-  e)s  —  q  +  eXs 
9  potentially,  policy  unstable  for  Q  <  1! 


If  {Qfc}J/=1  is  only  equitable  wrt  to  p>... 

9  per-vehicle  service  capacity  is  unbalanced  =>  policy  stable  but  not 

optimal 

9  guaranteed  to  be  within  m  of  optimal  unbiased  performance 
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Comments 


If  {Qk}k=i  's  only  equitable  wrt  to  p1/2... 

9  3  k  such  that  g-^  —  X  [1  / m  -\-  e)s  —  q  +  eXs 
9  potentially,  policy  unstable  for  Q  <  1! 
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Special  cases 


Case  s  =  0: 

9  stability  not  an  issue: 


generation  rate 


n 

m  TSPIength(n) 

' - v - ' 

service  rate 


demand  growth  rate 


9  since  TSPIength(n)  oc  yfn  =y  stability  for  all  A,  m 

9  equitability  only  wrt  to  <pV2  provides  optimal  performance 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT) 


Dynamic  Vehicle  Routing  (Lecture  4/8)  29junl0  @  Baltimore,  ACC 


25  /  27 


Special  cases 


Case  s  =  0: 

®  stability  not  an  issue: 


generation  rate 


n 

m  TSPIength(n) 

' - v - ' 

service  rate 


demand  growth  rate 


®  since  TSPIength(n)  oc  ^J~n  =>  stability  for  all  A,  m 

9  equitability  only  wrt  to  <pV2  provides  optimal  performance 
Case  (p  —  uniform: 

®  equitable  wrt  to  <p  =>  equitable  wrt  to  (p1/2 
9  no  need  to  use  algorithms  for  simultaneous  equitability 
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Workshop  Structure  and  Schedule 


8:00-8:30am 

8:30-9:00am 

9:05-9:50am 

Coffee  Brea 
Lecture  #1 
Lecture  #2 

k 

Intro  to  dynamic  vehicle  routing 

Prelims:  graphs,  TSPs  and  queues 

9:55-10:40am 

Lecture  #3 

The  single-vehicle  DVR  problem 

10:40-ll:00am 

ll:00-ll:45pm 

ll:45-l:10pm 

l:10-2:10pm 

2:15-3:00pm 

3:00-3:20pm 

3:20-4:20pm 

Break 

Lecture  #4: 
Lunch  Break 
Lecture  #5: 
Lecture  #6: 
Coffee  Break 
Lecture  #7: 

The  multi-vehicle  DVR  problem 

Extensions  to  vehicle  networks 
Extensions  to  different  demand  models 

Extensions  to  different  vehicle  models 

4:25-4:40pm 

4:45-5:00pm 

Lecture  #8: 

Extensions  to  different  task  models 

Final  open-floor  discussion 
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Lecture  outline 


Q  Territory  Partitioning 


Q  The  multi-vehicle  DVR  problem 


Q  Multi-vehicle  DVR  policies  based  on  partitioning 
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Dynamic  Vehicle  Routing  for  Robotic  Networks 
Lecture  #5:  Extensions  to  vehicle  networks  and 
distributed  algorithms 


Francesco  Bullo1  Emilio  Frazzoli2  Marco  Pavone2 

Ketan  Savla2  Stephen  L.  Smith2 


UCSB 


1CCDC 

University  of  California,  Santa  Barbara 
bulloOengineering . ucsb . edu 

2LIDS  and  CSAIL 

Massachusetts  Institute  of  Technology 
{frazzoli , pavone , ksavla , slsmith}@mit . edu 


Workshop  at  the  2010  American  Control  Conference 
Baltimore,  Maryland,  USA,  June  29,  2010,  8:30am  to  5:00pm 
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Territory  partitioning  via  centralized  space  planning 


UCSB  Campus  Development  Plan,  2008 
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Lecture  outline 


Q  Motivation  and  inspiration  from  biology 

Q  Intro  to  comm  models,  multi-agent  networks  and  distributed  algorithms 

Q  Partitioning  with  synchronous  proximity-graphs  communication 

Q  Partitioning  with  gossip  (asynchronous  pair-wise)  communication 

Q  Partitioning  with  no  explicit  inter-vehicle  communication 
o  No  explicit  communication  policy 
o  Game-theoretic  interpretation 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  5/8)  29junl0  ©  Baltimore,  ACC  2/44 


Territory  partitioning  akin  to  animal  territory  dynamics 


I 

/  r 


*  '  i  '  V 


Tilapia  mossambica,  “Hexagonal 
Territories,”  Barlow  et  al,  ’74 


Red  harvester  ants,  “Optimization,  Conflict,  and 
Nonoverlapping  Foraging  Ranges,”  Adler  et  al,  ’03 


Sage  sparrows,  “Territory  dynamics  in  a  sage  sparrows 
population,”  Petersen  et  al  ’87 
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Territory  partitioning:  behaviors  and  optimality 


DESIGN  of  performance  metrics 

Q  how  to  cover  a  region  with  n  minimum-radius  overlapping  disks? 

Q  how  to  design  a  minimum-distortion  (fixed-rate)  vector  quantizer? 
Q  where  to  place  mailboxes  in  a  city  /  cache  servers  on  the  internet? 

ANALYSIS  of  cooperative  distributed  behaviors 


how  do  animals  share  territory? 
how  do  they  decide  foraging 
ranges? 

how  do  they  decide  nest  locations? 

O  what  if  each  robot  goes  to  “center”  of  own  dominance  region? 
0  what  if  each  robot  moves  away  from  closest  vehicle? 
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Intro  to  communication  models,  multi-agent  networks  and 
distributed  algorithms 


References 

Q  I.  Suzuki  and  M.  Yamashita.  Distributed  anonymous  mobile  robots: 
Formation  of  geometric  patterns.  SIAM  Journal  on  Computing , 
28(4):1347-1363,  1999 

Q  N.  A.  Lynch.  Distributed  Algorithms.  Morgan  Kaufmann,  1997 

Q  D.  P.  Bertsekas  and  J.  N.  Tsitsiklis.  Parallel  and  Distributed  Computation: 
Numerical  Methods.  Athena  Scientific,  1997 

Q  S.  Martfnez,  F.  Bullo,  J.  Cortes,  and  E.  Frazzoli.  On  synchronous  robotic 
networks  -  Part  I:  Models,  tasks  and  complexity.  IEEE  Transactions  on 
Automatic  Control ,  52(12):2199-2213,  2007 

Objective 

Q  meaningful  +  tractable  model 
9  information/control/communication  tradeoffs 
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Lecture  outline 


Q  Intro  to  comm  models,  multi-agent  networks  and  distributed  algorithms 

Q  Partitioning  with  synchronous  proximity-graphs  communication 

Q  Partitioning  with  gossip  (asynchronous  pair-wise)  communication 

Q  Partitioning  with  no  explicit  inter-vehicle  communication 
«  No  explicit  communication  policy 
®  Game-theoretic  interpretation 
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Preliminary:  Processor  network  and  distributed  algorithm 


Processor  network:  group  of  processors  capable  to  exchange  messages 
along  edges  and  perform  local  computations 


Distributed  algorithm  for  a  network  of  processors  consists  of 

Q  WW,  the  processor  state  set 
Q  A,  the  communication  alphabet 
O  stfW  :  l/l/[;]  x  An  — >  l/l/t'l,  the  state-transition  map 
O  msgt'l  :  l/l/l']  — >  A,  the  message-generation  map 
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Robotic  network 


A  robotic  network  is 
Q  set  of  robots  moving  in  space  Q 
0  interaction  graph 


Disk,  visibility  and  Delauney  graphs 
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Communication  models  for  robotic  networks 


0  fixed,  directed,  balanced 
0  switching 

0  proximity/geometric  or 
state-dependent 

O  random,  random  geometric 
(packet  losses) 


O  message 
0  packet/bits 

Sensing  model 

0  absolute  coords  other  robots 

0  absolute  coords  environment 
boundary 
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Communication  models  for  robotic  networks 


0  fixed,  directed,  balanced  ®  message 

0  switching  ®  packet/bits 

O  proximity/geometric  or 
state-dependent 

O  random,  random  geometric 
(packet  losses) 
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Synchronous  control  and  communication 


0  communication  schedule 
0  communication  alphabet 
0  set  of  values  for  processor  vars 

0  message-generation  function 
0  state-transition  functions 
0  control  function 


T  =  {fy}t€N0  C  ®>0 
A 

1/1/ 

msg  :  T  x  Qx  W  —>  A 
stf  :  T  x  W  x  An  -►  1/1/ 

Ctrl  :!>0xQx  1/1/  x  AN  -»•  U 
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Lecture  outline 


Q  Partitioning  with  synchronous  proximity-graphs  communication 


No  explicit  communication  policy 
Game-theoretic  interpretation 


M.  Pavone,  A.  Arsie,  E.  Frazzoli,  and  F.  Bullo.  Equitable  partitioning  policies  for  mobile  robotic 
networks.  IEEE  Transactions  on  Automatic  Control,  2010.  (Submitted  Dec  2008  and  Aug  2009) 
to  appear 
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Spatially-distributed  policies  for  DVR 
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Median  Voronoi  diagrams  (and  beyond)  with  synchronous 
proximity-graphs  communication 


Voronoi+centering  law 

At  each  comm  round: 

1:  acquire  neighbors5  positions 
2:  compute  own  dominance  region 
3:  move  towards  center  of  own 
dominance  region 


S.  Martinez,  J.  Cortes,  and  F.  Bullo.  Motion  coordination  with  distributed  informa¬ 
tion.  IEEE  Control  Systems  Magazine,  27(4): 75-88,  2007 
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Experimental  Partitioning 


Takahide  Goto,  Takeshi  Hatanaka,  Masayuki  Fujita 
Tokyo  Institute  of  Technology 
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Hardware-in-the-loop  Partitioning  and  DVR  for  UAVs 


( 15x  real  time j 

□ 

D  4 

□ 


John  J.  Enright,  Chung  Hsieh,  Emilio  Frazzoli 
ARES  Group,  MIT  and  UCLA 
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Experimental  Partitioning 


Optimal  Distributed  Coverage  Control 
for  Multiple  Hovering  Robots  with 
Downward  Facing  Cameras 

Mac  Schwager 
Brian  Julian 
Daniela  Rus 

Distributed  Robots  Laboratory,  CSAIL 


Mac  Schwager,  Brian  Julian,  Daniela  Rus 
Distributed  Robots  Laboratory,  MIT 
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Equitable  and  median  Voronoi  diagrams  with  synchronous 
proximity-graphs  communication 


Distributed  algorithm  to  partition  the  workspace  according  to: 
Q  median  Voronoi  diagram  (relevant  in  light-load) 

O  equitable  (relevant  in  heavy  load) 


r Voronoi  Diagrams 

Voronoi  partition  {Vi,., 
points  (pi, . .  •  ,pm): 

V;  =  (x  e  Q  j  |x  -  pi\ 

. .  Vm\  generated  by 

I2  <  II*-p/II2,Y/V  '} 
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Partitioning  using  Power  Diagrams 

Power  distance 


®  p  =  (pi, . . . ,  pm)  collection  of  points  in  Q  Cl2 
®  each  pi  has  assigned  a  weight  w;  6  R 
«  power  distance  function  dp(x,  p,-;  w,)  =  ||x  —  p(-||2  —  w; 


Power  Diagrams 


Power  diagram  {Vi, ,  Vm}  generated  by 
weighted  points  ((pi,  wi), . . . ,  (pm,  wm)): 

a  Vi  =  {x  G  Q|||x  -  p/||2  -  Wj  ^ 
l|x  —  py||2  -  Wj,Vj  ^  /} 
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Equitable  and  median  Voronoi  diagrams  with  synchronous 
proximity-graphs  communication 


“Ambitious”  goal: 


Distributed  algorithm  to  partition  the  workspace  according  to: 
Q  median  Voronoi  diagram  (relevant  in  light-load) 

Q  equitable  (relevant  in  heavy  load) 


In  general,  an  equitable  Voronoi  Diagram  fails  to  exist... 
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Existence  theorem  for  Power  diagrams 


Existence  theorem 


Let  p  =  (pi, . . . ,  pm)  be  the  positions  of  m  >  1  distinct  points  in  Q.  Then 
there  exist  weights  (1/1/1, . . . ,  wm)  such  that  the  corresponding  Power 
diagram  is  equitable  with  respect  to  <p 
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Existence  theorem  for  Power  diagrams 


Existence  theorem 


Let  p  =  (pi, . . . ,  pm)  be  the  positions  of  m  >  1  distinct  points  in  Q.  Then 
there  exist  weights  (wi, . . . ,  wm)  such  that  the  corresponding  Power 
diagram  is  equitable  with  respect  to  ip 


V1  V2 


Q - Q 
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Existence  theorem 


Let  p  =  (pi, . . . ,  pm)  be  the  positions  of  m  >  1  distinct  points  in  Q.  Then 
there  exist  weights  (wi, . . . ,  wm)  such  that  the  corresponding  Power 
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Existence  theorem  for  Power  diagrams 


Existence  theorem 


Let  p  —  (pi, . . . ,  pm)  be  the  positions  of  m  >  1  distinct  points  in  Q.  Then 
there  exist  weights  (u/i, . . . ,  wm)  such  that  the  corresponding  Power 
diagram  is  equitable  with  respect  to  p 
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Gradient  descent  law  for  equitable  partitioning 


o  Wj  locally  controlled  by  vehicle  / 
o  locational  optimization  function 

n(w)  =  J2(  v(x)dx)  =  J^|v//(m/)|~1 

/=1  '  /=1 

»  spatially-distributed  gradient:  =  E/e/v,  (jv^r  -  y^r) 


Gradient  law  for  equitable  partitioning 

At  each  comm  round: 

1:  acquire  neighbors  *  positions 
2:  compute  own  dominance  region 
3:  Wi  <-  Wi 
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Including  the  median  Voronoi  diagram  property 


Close  to  Voronoi: 

®  basic  idea:  keep  the  weights  close  to  zero 
®  modify  the  gradient  descent  law  as 


dH  dH  .  dH 

w/  =  -- - wi,  —  pi-  — 

owj  op/  owj 


Wi  =  0 


Motion  toward  the  median: 

®  basic  idea:  add  a  term  that  enforces  computation  of  the  median 
®  gradient  term  for  computation  of  the  median: 


d7if\/\/ 

dp-, 


Pi  ~  x 
Pi  ~  x 


(p(x)dx 


®  modify  the  gradient  descent  law  as 

dH  .  <WFw  ...  (dH  cWFw\ 

Wi  =  --^r  p' “  -g^r) 
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Convergence  result 


Theorem  (Convergence) 


Assume  that  the  pi 's  are  distinct.  Then,  the  w;  's  converge  asymptotically 
to  a  vector  of  weights  that  yields  an  equitable  Power  diagram 


®  guaranteed  convergence  for  any  set  of  distinct  points 

=*►  global  convergence  result 

®  distributed  over  the  dual  graph  of  the  induced  Power  diagram 

communication,  on  average,  with  six  neighbors 

®  adjusting  the  weights  sufficient  to  obtain  an  equitable  diagram 

=*>  move  the  p,’s  to  optimize  secondary  objectives 
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Simulation 


Region  partition: 

Min  Area 

Density 

Uniform  J 


►  Quit 


Ready 
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Lecture  outline 


Q  Partitioning  with  gossip  (asynchronous  pair-wise)  communication 

Q  Partitioning  with  no  explicit  inter-vehicle  communication 
e  No  explicit  communication  policy 
o  Game-theoretic  interpretation 
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Gossip  (asynchronous  pair-wise)  partitioning  policy 


Q  Random  communication  between  two  regions 
Q  Compute  two  centers 
O  Compute  bisector  of  centers 
O  Partition  two  regions  by  bisector 


F.  Bullo,  R.  Carli,  and  P.  Frasca.  Gossip  coverage  control  for  robotic  networks:  Dynam¬ 
ical  systems  on  the  the  space  of  partitions.  SIAM  Review,  January  2010.  Submitted 
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Peer-to-peer  convergence  analysis  (proof  sketch  1/3) 

Lyapunov  function  for  peer-to-peer  territory  partitioning 


n(v)  =  E  center(v'/)  -  q\\)<K<i)d<i 

»■= i Jv' 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  5/8)  29junl0  @  Baltimore,  ACC  29  /  4 A 


The  space  of  partitions  (proof  sketch  2/3) 

Definition  (Space  of  finitely-convex  partitions) 


Fix  £,  the  set  v  is  collections  of  n  subsets  of  Q,  (vi, . . . ,  vn},  such  that 
Q  vi  U  •  •  •  U  vn  =  Q, 

Q  interior(v;)  fl  i nterior( vy)  =  0  if  i  ^  j,  and 
O  each  v,-  is  union  of  £  convex  sets 


Given  sets  A  and  B,  symmetric  distance  is: 

dA(A,  B)  =  area  ({A(J  B)  \  ( AnB )) 


Theorem  (topological  properties  of  the  space  of  finitely-convex  partitions) 


Partition  space  with  (u,  v)  i— >  ^3/Li  (u/,  vi)  is  metric  and  compact 
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Peer-to-peer  convergence  analysis  (proof  sketch  1/3) 


O  state  space  is  not  finite-dimensional 
non-convex  disconnected  polygons 
arbitrary  number  of  vertices 
9  peer-to-peer  map  is  not  deterministic,  ill-defined  and  discontinuous 
two  regions  could  have  same  centers 
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Convergence  with  persistent  switches  (proof  sketch  3/3) 


o  X  is  metric  space 

o  finite  collection  of  maps  7~;  :  X  — >  X  for  /  e  / 
o  consider  sequences  {x^>o  C  X  with 

xm  =  Ti(t)(xe) 

Assume: 

Q  W  C  X  compact  and  positively  invariant  for  each  T; 

Q  U  \  W  —*M.  decreasing  along  each  7"; 

O  U  and  T,-  are  continuous  on  1/1/ 

Q  there  exists  probability  p  e  ]0, 1[  such  that,  for  all  indices  /  e  /  and 
times  /,  we  have  Prob  [x^+i  =  T(x^)  |  past]  >  p 

If  xo  6  IV,  then  almost  surely 

X£  — »■  (intersection  of  sets  of  fixed  points  of  all  T;)  fl  L/-1(c) 
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Lecture  outline 


Q  Motivation  and  inspiration  from  biology 

Q  Intro  to  comm  models,  multi-agent  networks  and  distributed  algorithms 

Q  Partitioning  with  synchronous  proximity-graphs  communication 

^  Partitioning  with  gossip  (asynchronous  pair-wise)  communication 

Q  Partitioning  with  no  explicit  inter-vehicle  communication 
o  No  explicit  communication  policy 

«  Game-theoretic  interpretation 


A.  Arsie,  K.  Savla,  and  E.  Frazzoli.  Efficient  routing  algorithms  for  multiple  vehicles  with  no  explicit 
communications.  IEEE  Transactions  on  Automatic  Control,  54(10):2302-2317,  2009 
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Motivation 


®  Similar  result  using  the  gossip  partitioning  policy 


Salient  Features 


o  Explicit  agent-to-agent 
communication 

o  Needs  knowledge  of  p 
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Motivation 


®  Similar  result  using  the  gossip  partitioning  policy 
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Partitioning  with  no  explicit  inter-vehicle  communication 


Inspiration:  Distributed  MacQueen  algorithm 


o  Pick  any  m  generator  points  (pi, . . . ,  pm )  £  Qm 
o  Iteratively  sample  points  qj  according  to  probability  density  function  p 
At  each  iteration  j: 

3  Assign  the  sampled  point  to  the  nearest  generator  /*(q/)  £  {1, . . . ,  m} 

3  update  the  position  of  generator  /*  as 

(#pts  assigned  in  past)  pi*  +  qj 
P'  #pts  assigned  in  past  +  1 
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Algorithms 


No  sensor  policy 


For  all  time  t,  each  vehicle  moves  towards: 
o  the  nearest  outstanding  task;  else, 

o  the  (nearest)  point  minimizing  the  average 
distance  to  tasks  serviced  in  the  past 
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Illustration 
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Algorithms 


No  sensor  policy 


For  all  time  t,  each  vehicle  moves  towards: 
o  the  nearest  outstanding  task;  else, 

o  the  (nearest)  point  minimizing  the  average 
distance  to  tasks  serviced  in  the  past 


Sensor-based  policy 


For  all  time  t,  each  vehicle  moves  towards: 

®  the  nearest  among  outstanding  tasks  that 
is  closest  to  it  than  other  vehicles;  else, 

o  the  (nearest)  point  minimizing  the  average 
distance  to  tasks  serviced  in  the  past 
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Differences  with  the  MacQueen  algorithm 


9  At  each  iteration,  the  no-communication  algorithm  computes  the 
”  Fermat-Weber  (FW)  point”  with  respect  to  the  set  of  tasks  serviced 
by  a  vehicle;  MacQueen  algorithm  computes  the  mean 

FW/  =  argminp.eQ  ^  \\q  -  p(- 

qGpast  tasks,- 

Meani  =  |past  tasks,|  £  q 

qC past  tasks,- 

o  No  simple  recursion  like  the  MacQueen  algorithm  — >  need  to  store 
locations  of  all  the  tasks  serviced  in  the  past 

o  Sequence  of  FW  points  exhibit  more  complex  behavior  than  the 
sequence  of  means. 
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Analysis  of  the  algorithm 


®  p;(t):  loitering  location  of  agent  /  at  time  t 
®  Sufficient  to  study  convergence  of  (pi(t), . . . ,  pm(t)) 
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«  No  explicit  communication  policy 

o  Game-theoretic  interpretation 
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Coverage  as  a  geometric  game 


Strategies 


®  P  =  (pi,...,pm)  e  Qm 

o  When  a  new  task  is  generated,  every  vehicle  move  towards  its  location 
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Coverage  as  a  geometric  game 


Strategies 


®  P  =  (pi,...,pm)  e  Qm 

o  When  a  new  task  is  generated,  every  vehicle  move  towards  its  location 


Utility  Function 


o  Upon  its  generation,  each  task  offers  continuous  reward  at  rate  unity 

o  A  task  expires  as  soon  as  two  vehicles  are  present  at  its  location  or 
after  diam(Q)  time,  whichever  occurs  first. 

o  Utility  function:  expected  time  spent  alone  at  the  next  task  location 

Ui(pi,  p-i )  =  E v[Ri(p,  q )]  1  Ep 
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Properties  of  the  Game 


»  Potential  function:  ip(p)  m  -  J2?=i  ,{Vi(P)  II Pi  ~  pIIpCpMp 
»  The  coverage  spatial  game  is  a  potential  game 
(W;(p)  =  V’(P)  -  V’(P-z)) 

9  U  is  a  Wonderful  Life  utility  function 


max 


0,  min  || pj  -  q ||  -  ||p;  -  q\ 
j¥=' 


[  Characterization  of  Equilibria 

critical  point  of  H 

=4>  pure  Nash  equilibrium 
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Properties  of  the  Game 


®  Potential  function:  'ip(p)  =  -  J2?=i  fVi(p)  IIP/  -  q\\(p(q)dq 
®  The  coverage  spatial  game  is  a  potential  game 
(W;(p)  =  ^(p)  -  'ip(p-i)) 

9  U  is  a  Wonderful  Life  utility  function 
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No  communication  policy  as  a  learning  algorithm 


[Complete  Information 

Pi  -  ddpMi{p)  -  .fvj(p)  \\p,-qq\\  P{q)dq  ~ 

=>  gradient  descent  policy 
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No  communication  policy  as  a  learning  algorithm 
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Lecture  outline 


Q  Motivation  and  inspiration  from  biology 

Q  Intro  to  comm  models,  multi-agent  networks  and  distributed  algorithms 

Q  Partitioning  with  synchronous  proximity-graphs  communication 

Q  Partitioning  with  gossip  (asynchronous  pair-wise)  communication 

Q  Partitioning  with  no  explicit  inter-vehicle  communication 
o  No  explicit  communication  policy 
o  Game-theoretic  interpretation 
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No  communication  policy  as  a  learning  algorithm 
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Workshop  Structure  and  Schedule 


8:00-8:30am 

Coffee  Break 

8:30-9:00am 

Lecture  #1: 

9:05-9:50am 

Lecture  #2: 

9:55-10:40am 

Lecture  #3: 

10:40-ll:00am 

Break 

ll:00-ll:45pm 

Lecture  #4: 

ll:45-l:10pm 

Lunch  Break 

l:10-2:10pm 

Lecture  #5: 

2:15-3:00pm 

Lecture  #6: 

3:00-3:20pm 

Coffee  Break 

3:20-4:20pm 

Lecture  #7: 

4:25-4:40pm 

4:45-5:00pm 

Lecture  #8: 

Intro  to  dynamic  vehicle  routing 
Prelims:  graphs,  TSPs  and  queues 
The  single-vehicle  DVR  problem 

The  multi-vehicle  DVR  problem 

Extensions  to  vehicle  networks 
Extensions  to  different  demand  models 

Extensions  to  different  vehicle  models 
Extensions  to  different  task  models 
Final  open-floor  discussion 
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Lecture  outline 


Q  Stochastic  Time  Constraints 

9  Policy  Independent  Lower  Bound 
9  Nearest  Depot  Assignment  Policy 
9  Batch  Policy 


^  Priority  Classes  of  Demands 

9  Policy  Independent  Lower  Bound 
9  Separate  Queues  Policy 
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Motivation:  Time-Critical  Tasks 


Motivating  Scenario 

9  Group  of  UAVs  equipped  with  sensors,  monitoring  region 
9  Alerted  of  events  that  require  close-range  observation 

Events  with  time  constraints: 

9  Each  event  must  be  observed  within  a  time-window 

Events  with  priority  levels: 

9  Each  event  has  associated  level  of  importance  (e.g.  1  to  10) 
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Lecture  outline 


Q  Stochastic  Time  Constraints 

9  Policy  Independent  Lower  Bound 
9  Nearest  Depot  Assignment  Policy 
9  Batch  Policy 

^  Priority  Classes  of  Demands 

9  Policy  Independent  Lower  Bound 
9  Separate  Queues  Policy 

M.  Pavone  and  E.  Frazzoli.  Dynamic  vehicle  routing  with  stochastic  time  constraints.  In  IEEE  Int. 
Conf.  on  Robotics  and  Automation,  Anchorage,  AK,  May  2010 

M.  Pavone,  N.  Bisnik,  E.  Frazzoli,  and  V.  Isler.  A  stochastic  and  dynamic  vehicle  routing  problem 
with  time  windows  and  customer  impatience.  ACM/Springer  Journal  of  Mobile  Networks  and 
Applications,  14(3):350-364,  2009 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  6/8)  29junl0  ©  Baltimore,  ACC  4/23 


DVR  with  stochastic  time  constraints 


Model: 

®  basic  DVR  model  + 

®  demand  j  active  for  a  random  patience 
time  Gj 

®  Gj' s  i.i.d.  sequence  rsj  Fg 
o  demand  j  expires  if  not  serviced  within  Gj 

Service  constraint: 

o  lim^+ooP^  [Wj  <  Gj]:  acceptance  probability  for  policy  i r 
®  (j)d  G  (0,  1):  desired  acceptance  probability 

•  constraint:  limy-H-ooP*.  [Wj  <  Gj]  >  #' 
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Problem  formulation 


Proof  sketch: 

®  main  idea:  theory  of  regenerative  processes 

®  regeneration  points:  times  a  new  demand  finds  the  system  empty 
®  expected  length  of  busy  cycles  is  finite 
®  use  classic  limit  theorems 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT) 


Dynamic  Vehicle  Routing  (Lecture  6/8)  29junl0  ©  Baltimore,  ACC 


6/23 


Problem  formulation 


Problem  statement 


Solve  problem  OPT : 


min  |7r|,  subject  to  lim  [Wj  <  Gj]  > 

7T  J^OO 
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Lower  bound 


Intuition  for  lower  bound: 


F[Wj  <  Gj]  < 
< 


mi 


\\Xj-Xk\\ 


<  G; 


sup  P 

{Pl,-,Pm)eQm 


m 


\\Xj-Pk\\ 


<  Gj 


=n(p1,...,pm) 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT) 


Dynamic  Vehicle  Routing  (Lecture  6/8)  29junl0  ©  Baltimore,  ACC 


7/23 


Lower  bound 


Intuition  for  lower  bound: 


P  [Wj<  Gj ]  < 


mi 


l|X/-Xfc|| 


<  G; 


<  sup  P 


\\Xj  Pk II  ^  r 

mm/ce{l,...,rn}  - v -  <  GJ 


=n(pi,...,Pm) 
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NDA  policy  (optimal  as  A  — >  0) 


Nearest  Depot  Assignment  (NDA)  policy 

Compute  maximum  of  Tt :  (pi, . . . ,  pm ) . 

>' 

> ' 

>a 

Then: 

■ 

■ 

■ 

■ 

l:  ft  is  depot  of  kth  vehicle 

> 

> 

> 

> 

2:  nearest-depot  assignment 

■ 

i 

■ 

■ 

3:  FCFS  service 

> 

> 

> 

FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  6/8)  29junl0  ©  Baltimore,  ACC  8/23 


Lower  bound 


Intuition  for  lower  bound 

P  [Wj  <  Gj]  < 


<  GJ 


<  sup  P 

(pi,...,pm)eQms_ 


m 


\\Xj-Pk\\ 


<  Gj 


=7-i{Pl,-,Pm) 


Lower  bound 

OPT  is  lower  bounded  by: 

OPT  :  min 

m<GN>0 

m 

S.t. 

sup  TC(pi,. 

(pi,-,Pm)eQm 

•  •  5  Pm)  if  ^ 

Devised  algorithms  to  solve  OPT_ 
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NDA  policy  (optimal  as  A  — >  0) 


Nearest  Depot  Assignment  (NDA)  policy 

Compute  maximum  of  7 ~i :  (pi, . . . ,  pm ) . 

>' 

> ' 

Then: 

■ 

■ 

■ 

■ 

l:  ft  is  depot  of  kth  vehicle 

> 

> 

> 

> 

2:  nearest-depot  assignment 

■ 

i 

■ 

■ 

3:  FCFS  service 

> 

> 

> 

Proof  sketch: 

®  as  usual,  as  A  — ►  0+,  the  problem  reduces  to  optimal  pre-positioning 
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NDA  policy  (optimal  as  A  — >  0) 


Nearest  Depot  Assignment  (NDA)  policy 

Compute  maximum  of  H :  (pi, . . . ,  pm ) . 

>' 

> ' 

>a 

Then: 

■ 

■ 

■ 

■ 

l:  ft  is  depot  of  /eth  vehicle 

> 

> 

> 

> 

2:  nearest-depot  assignment 

■ 

i 

■ 

■ 

3:  FCFS  service 

> 

> 

> 

Proof  sketch: 

®  as  usual,  as  A  — >  0+,  the  problem  reduces  to  optimal  pre-positioning 


Characterization  of  batch  policy 


Proof  sketch  (m=l): 

o  upper  bound  expected  length  of  TSP  tour  with  const  •  A/m2,  via 
control-theoretical  methods 

9  use  Markov’s  ineq  to  lower  bound: 


P  [W  <  G]  >  P  [W  <  G\2  TSP  <  0]P  [2  TSP  <  9] 
>  (1  -  FG(0))(1  -E[2TSP]/0) 


> 

r 

- 

- 

- 

- 

- 

- 
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Batch  policy 


Batch  (B)  policy 

Partition  Q  into  m  simultaneously 
equitable  subregions  and  assign  one 
vehicle  to  each  subregion.  Then: 

1:  each  vehicle  services  demands  by 
forming  TSP  tours 


Performance  of  batch  policy 

a  if  s=0:  /t?b  =  min<j 
a  with  time  window: 

s:  a 

sup*6R>D(l-FG(#))(l-^)>^j 
t?b / m*  <  3.78,  when  A  large  and  (f)d  — 

\ 

.  1“ 
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Lecture  outline 


Policy  Independent  Lower  Bound 
Nearest  Depot  Assignment  Policy 
Batch  Policy 


^  Priority  Classes  of  Demands 

9  Policy  Independent  Lower  Bound 
9  Separate  Queues  Policy 
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Demands  with  priority  levels 


9  m  vehicles 
o  n  classes  of  demands 
9  1  =  highest  priority 
9  n  =  lowest  priority 

o  Poisson  arrivals  Ai, . . . ,  An 

o  locations  uniformly  distributed 
can  extend  to  non-uniform  p 


Steady-state  expected  system-time  Ti, . . . ,  Tn 


[  Goal  for  vehicles 

1 

Minimize  c\T \  +  •  •  •  +  cnTn 

(1  Cj  priority  of  class  /') 

S.  L.  Smith,  M.  Pavone,  F.  Bullo,  and  E.  Frazzoli.  Dynamic  vehicle  routing  with  priority  classes  of 
stochastic  demands.  SIAM  Journal  on  Control  and  Optimization,  48(5):3224-3245,  2010 
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Demands  with  priority  levels 


o  m  vehicles 
<*  n  classes  of  demands 
9  1  =  highest  priority 
9  n  =  lowest  priority 

o  Poisson  arrivals  Ai, . . . ,  An 

o  locations  uniformly  distributed 
can  extend  to  non-uniform  p 


Steady-state  expected  system-time  7"i, . . . ,  Tn 


\  Goal  for  vehicles 

1 

Minimize  c\T \  +  •  •  •  +  cnTn 

(1  Cj  priority  of  class  /') 

S.  L.  Smith,  M.  Pavone,  F.  Bullo,  and  E.  Frazzoli.  Dynamic  vehicle  routing  with  priority  classes  of 
stochastic  demands.  SIAM  Journal  on  Control  and  Optimization,  48(5):3224-3245,  2010 
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Demands  with  priority  levels 


9  m  vehicles 
o  n  classes  of  demands 
9  1  =  highest  priority 
9  n  =  lowest  priority 

o  Poisson  arrivals  Ai, . . . ,  A„ 

o  locations  uniformly  distributed 
can  extend  to  non-uniform  p 

Steady-state  expected  system-time  7~i, 


Goal  for  vehicles 


Minimize  c\  7~i  +  •••  +  cnTn  (|  c;  =>t  priority  of  class /) 


S.  L.  Smith,  M.  Pavone,  F.  Bullo,  and  E.  Frazzoli.  Dynamic  vehicle  routing  with  priority  classes  of 
stochastic  demands.  SIAM  Journal  on  Control  and  Optimization,  48(5):3224-3245,  2010 
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Demands  with  priority  levels 


o  m  vehicles 
o  n  classes  of  demands 
9  1  =  highest  priority 
9  n  =  lowest  priority 

o  Poisson  arrivals  Ai, . . . ,  A„ 

o  locations  uniformly  distributed 
can  extend  to  non-uniform  p 


Steady-state  expected  system-time  T i, . 

Minimize  c\  T\  +  •••  +  cnTn  (t  c;  =>t  priority  of  class /) 


S.  L.  Smith,  M.  Pavone,  F.  Bullo,  and  E.  Frazzoli.  Dynamic  vehicle  routing  with  priority  classes  of 
stochastic  demands.  SIAM  Journal  on  Control  and  Optimization,  48(5):3224-3245,  2010 
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Literature  Review 


Classic  Priority  Queueing 

L.  Kleinrock.  Queueing  Systems.  Volume  II:  Computer  Applications.  Wiley,  New 
York,  1976 

E.  G.  Coffman  Jr.  and  I.  Mitrani.  A  characterization  of  waiting  time  performance 
realizable  by  single-server  queues.  Operations  Research,  28(3):810-821,  1980 


Related  Combinatorial  Problems 

A.  Blum,  P.  Chalasani,  D.  Coppersmith,  B.  Pul leyblank,  P.  Raghavan,  and  M.  Su¬ 
dan.  The  minimum  latency  problem.  In  ACM  Symposium  on  the  Theory  of  Com¬ 
puting,  pages  163-171,  Montreal,  Canada,  1994 

M.  Z.  Spivey  and  W.  B.  Powell.  The  dynamic  assignment  problem.  Transportation 
Science,  38(4):399-419,  2004 

A.  Blum,  S.  Chawla,  D.  R.  Karger,  T.  Lane,  A.  Meyerson,  and  M.  Minkoff.  Ap¬ 
proximation  algorithms  for  orienteering  and  discounted-reward  TSP.  SIAM  Journal 
on  Computing,  37(2):653-670,  2007 
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Light  load 


In  light  load: 

9  Each  vehicle  can  return  to  a  median  between  arrivals 
9  Priority  levels  do  not  change  behavior. 

Optimal  solution: 

m  vehicle  SQM  policy  is  optimal  (or  an  adaptive  policy) 


m  Stochastic  Queueing  Median  (m-SQM) 

Compute  m-median  locations  and  assign  one  vehicle  to 
each  location. 

Then: 

1:  service  demands  in  FCFS  order 

2:  return  to  median  after  each  service  is  completed 
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Load  Factor  and  Stability 


Stable:  Queue  remains  bounded 
Define  oad  factor  as 

AiSi  +  •  •  •  +  Xnsn 

Q  := - 

m 

o  A;  —  arrival  rate  for  class  / 

®  s;  =  average  on-site  service  time  for  class  / 

As  before,  necessary  stability  condition  is  g  <  1 

Two  asymptotic  regimes 

Q  Light  load  g  — »  0+ 

Q  Heavy  load  g  — >  1~ 
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Lower  Bound  in  Heavy  Load 


Let  7*  =  optimal  value  of  cost  c\  T\  +  •  •  •  +  cnTn. 


Lower  bound  for  every  policy 

y*  ^  /^TSP I  Q I  j 

c  -  2m2 v2(l  -g)2^  1 

a=l 

(  n  \ 
Ca  + 2  ^2  CJ\ 

\  j=OL+l  J 

^ a 

Problem  parameters: 

9  arrival  rates  Ai, An  9  environment  area  \Q\ 

9  weights  ci, . . . ,  cn  9  vehicle  speed  v 

9  number  of  vehicles  m 
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Proof  Idea  of  Lower  Bound 


®  Allow  remote  service  of  some  classes:  ra  e  {0, 1}  for  each  class  a 
o  travel  distance  is  rada 


di/v 


demand  served  prior 


<D 

Si 


®  For  stability:  ( I'idi/v  +  s,-)  <  m 

®  Can  bound  travel  distance  as 


5  >  ftpp  /joT 

V2VE  ,nN, 

o  generates  a  linear  program  with  2n  —  1  constraints,  one  for  each 
combination  {ri, . . . ,  rn} 

9  solution  to  LP  is  largest  lower  bound 
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Separate  Queues  Performance 


f  Heavy  load  performance 

For  the  SQ  policy, 

7c,Sq  <2n2 

T*c  ~ 

as  q  — >  1~. 

o  n  =  number  of  classes 
o  independent  of  g:  c,  s,  A 


Heuristic  Improvements: 

O  Receding  horizon:  service  only  a  fraction  77  of  TSP 
Q  when  following  TSP,  service  newly  arrived  demands  within  e  of  TSP. 


6 


/ 


ELi  «. 


where  /i  is  fractional  in  tour  length  (i.e.,  0.1  for  10%  increase) 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT) 


Dynamic  Vehicle  Routing  (Lecture  6/8)  29junl0  ©  Baltimore,  ACC 


19  /  23 


Separate  Queues  Policy 


Input:  Probability  distribution  p  =  [pi, . . . ,  pn]. 

Separate  Queues  Policy 

Partition  environment  into  m  equal  area  regions  and 
assign  one  vehicle  to  each  region. 

Then: 

1:  Select  a  class  according  to  probability  dist  p 
2:  Service  all  demands  of  selected  class  following  TSP 
3:  Repeat 

Policy  performance  optimized  over  p. 
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Simulation  of  Separate  Queues  Policy 


Simulation: 

®  class  1  =  yellow 
®  class  2  =  grey 
o  ci  =  0.8  and  C2  =  0.2 

®  p  =  [0.82,0.18] 
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Proof  idea  for  upper  bound 


®  In  heavy-load,  shortest  path  through  N  points: 

=  /3tsp \/\Q\N  with  prob.  1  (BHH  theorem) 

®  Study  expected  #  of  outstanding  demands  at  each  iteration 
A/;(t  +  1)  <  f(Ni{t), Nm(t),  p,  A,  s) 


®  Function  f  has  a  linear  part  plus  a  sub-linear  part 
®  Bound  evolution  by  stable  linear  system  for  a  H  Q  <  1 

J\f(t  +  1)  =  A(p,  A,  s)AT(t)  +  B(p,  A,  s) 

®  Allows  computation  of  lim  supt^+00  W) 

®  Apply  Little’s  theorem  N;  —  A;  7” / 
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Workshop  Structure  and  Schedule 


8:00-8:30am 

8:30-9:00am 

9:05-9:50am 

Coffee  Brea 
Lecture  #1 
Lecture  #2 

k 

Intro  to  dynamic  vehicle  routing 

Prelims:  graphs,  TSPs  and  queues 

9:55-10:40am 

Lecture  #3 

The  single-vehicle  DVR  problem 

10:40-ll:00am 

ll:00-ll:45pm 

ll:45-l:10pm 

l:10-2:10pm 

2:15-3:00pm 

3:00-3:20pm 

3:20-4:20pm 

Break 

Lecture  #4: 
Lunch  Break 
Lecture  #5: 
Lecture  #6: 
Coffee  Break 
Lecture  #7: 

The  multi-vehicle  DVR  problem 

Extensions  to  vehicle  networks 
Extensions  to  different  demand  models 

Extensions  to  different  vehicle  models 

4:25-4:40pm 

4:45-5:00pm 

Lecture  #8: 

Extensions  to  different  task  models 

Final  open-floor  discussion 
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Lecture  outline 


Q  Stochastic  Time  Constraints 

®  Policy  Independent  Lower  Bound 
®  Nearest  Depot  Assignment  Policy 
®  Batch  Policy 

^  Priority  Classes  of  Demands 

®  Policy  Independent  Lower  Bound 
®  Separate  Queues  Policy 
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Dynamic  Vehicle  Routing  for  Robotic  Networks 
Lecture  #7:  Vehicle  Models 
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Vehicle  routing  with  differential  constraints 


o  What  happens  if  the  vehicles  are  subject  to  non-integrable  differential 
constraints  on  their  motion? 

o  Minimum  turn  radius,  constant  speed  (UAVs,  Dubins  cars) 
o  Minimum  turn  radius,  able  to  reverse  (Reeds-Shepps  cars) 
o  Differential  drive  robots  (e.g.,  tanks). 

o  Bounded  acceleration  vehicles  (e.g.,  helicopters,  spacecraft). 


o  Fundamentally  different  problems,  combining  combinatorial  task 
specifications  with  differential  geometry  and  optimal  control. 


o  Decompose  the  problem,  study  the  asymptotic  cases: 
o  Heavy  load:  Traveling  salesperson  problems, 
o  Light  load:  optimal  loitering  "stations" . 
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Outline  of  the  lecture 


^  Models  of  vehicles  with  differential  constraints 
^  Traveling  salesperson  problems 
Q  The  heavy  load  case 
Q  The  light  load  case 
o  Phase  transition  in  the  light  load 
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Models  of  vehicles  with  differential  constraints 


Reeds-Shepp  car 


x  =  v  cos  9 
y  —  vsin# 

6  =  ijj 

ve{-l,l} 

M  <  i /  p 


r  ^ 

Dubins  vehicle 

x  =  cos  9 

y  —  sin# 

9  =  uj 

cu  <  1/p 

Differential  drive 


x  =  ^  (u\  +  ojr)  cos  6 
y  =  \  (cji  +  ojr )  sin  6 
o  =  -p{ur-  U)I ) 

\ui\  <  l;  \(dr\  <  l 


[  Double  integrator 

X  —  u 

ii^ii  < 1 

||u||  <  1 
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DTRP  formulation 


Problem  setup 


o  m  identical  vehicles  in  Q 

o  Spatio-temporal  Poisson 
process:  rate  A  and  uniform 
spatial  density 

o  On-site  service  time  s  =  0 


o 

o 
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Stabilizability 


O 


task  generation  rate 


m 


n:  #  outstanding  tasks 


n 

TSPIength(/7) 

- V - s 

task  service  rate 


task  growth  rate 


®  TSPIength(n)  strictly  sub-linear 


stability  VA,  m 
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DTRP  formulation 


o 

o 


Problem  setup 


o  m  identical  vehicles  in  Q 

o  Spatio-temporal  Poisson 
process:  rate  A  and  uniform 
spatial  density 

o  On-site  service  time  s  =  0 


Objective 


o  Control  policy  7r  =  {task  assignment,  scheduling,  loitering} 
o  Tn  :=  lim  sup/^^  E[wait  time  of  task  /];  7*  =  inf^  Tn 
o  Design  n  for  which  T n  is  equal  to  or  within  a  constant  factor  of  7* 
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I 


o 

o 


Stabilizability 


task  generation  rate 


—  m  ■ 


TSPIength(n) 


=  task  growth  rate 


rr.  #  outstanding  tasks 


task  service  rate 


TSPIength(n)  strictly  sub-linear  =>  stability  VA,  m 


Euclidean  TSPIength(n)  =  ©(n1/2)  (Beardwood  et.  al.  ’59) 


®  Euclidean  TSP  based  path  planning  heuristic  =>  O(n) 


o  Traveling  salesperson  problems  for  differential  vehicles. 
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Outline  of  the  lecture 


Q  Traveling  salesperson  problems 
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Traveling  Salesperson  Problem 


Problem  Statement 


Find  the  shortest  closed  curve 
feasible  for  the  vehicle  through  a 
given  finite  set  of  points  in  the  plane 


®  NP-hardness  a  consequence  of  the  NP-hardness  of  the  Euclidean  TSP. 

®  Does  the  cost  of  this  TSP  increase  SUBLINEARLY  with  n? 

®  Is  there  a  polynomial-time  algorithm  that  returns  a  tour  of  length 
o(n)?? 

®  What  is  the  quality  of  the  solution? 
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Traveling  Salesperson  Problem 


Problem  Statement 


Find  the  shortest  closed  curve 
feasible  for  the  vehicle  through  a 
given  finite  set  of  points  in  the  plane 
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Stochastic  TSP:  A  nearest-neighbor  lower  bound 


Outline  of  the  calculations 


o  Calculate  (an  upper  bound  on)  expected  distance  from  an  arbitrary  vehicle 
configuration  to  closest  point,  5* 

o  Calculate  (an  upper  bound  on)  the  area  of  the  set  reachable  with  a  path  of 
length  5,  1Z$- 

O  Pr(£*  >  8)  >  max{0, 1  -  n\1Zs\/\Q\} 

o  Expected  length  of  the  tour  cannot  be  less  than  n  times  E[5*] 
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Towards  an  upper  bound:  tiling  based  algorithms 


9  The  way  the  ETSP  tours  are  constructed  relies  on  the  scaling 

properties  of  tours:  the  length  of  the  tour  scales  as  the  coordinates  of 
the  points. 
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Towards  an  upper  bound:  tiling  based  algorithms 


9  The  way  the  ETSP  tours  are  constructed  relies  on  the  scaling 

properties  of  tours:  the  length  of  the  tour  scales  as  the  coordinates  of 
the  points. 


9  No  such  scaling  exists  for  the  TSP  for  vehicles  with  differential 
constraints,  e.g.,  the  bound  on  the  curvature  for  the  Dubins  vehicle 
does  not  scale  with  the  coordinates  of  the  points! 

9  Any  tiling-based  algorithm  must  account  for  a  "preferential 
direction”,  e.g.,  by  penalizing  turning  for  Dubins  vehicles 
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Bead  construction 


Bead  properties 


®  Length(p_,  q,  p+ )  <  £  +  o(£2)  for  all  q  &  B 
9  Width:  w(t)  =  g  +  o{£3) 

9  The  beads  tile  the  plane 

9  Useful  for  Dubins  vehicle,  Reeds-Shepp  car  and  double  integrator 
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The  single-sweep  tiling  algorithm 


o  Tile  the  region  with  beads 

a  Sweep  the  bead  rows,  while  servicing  all  the  targets  in  every  bead  as 
follows: 
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Bead  construction 


Bead  properties 


9  Length(p_,  q,  p+)  <  £  +  o(£2)  for  all  q  &  B 
9  Width:  w(£)  =  g  +  o{£3) 

9  The  beads  tile  the  plane 

9  Useful  for  Dubins  vehicle,  Reeds-Shepp  car  and  double  integrator 


9  Diamond-like  cell  for  differential  drive 
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The  single-sweep  tiling  algorithm 


9  Tile  the  region  with  beads 

o  Sweep  the  bead  rows,  while  servicing  all  the  targets  in  every  bead  as 
follows: 

9  Service  every  task  q  in  £>_  using  the  ”p_  — ►  q  — >  p_”  protocol 
9  Move  from  p_  to  p+ 

9  Service  every  task  q  in  £>+  using  the  ”p+  — >  q  — »  p+"  protocol 
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Analysis  of  the  single-sweep  tiling  algorithm 

Path  length  calculations 


TSP(n)  —  (bead  row  length  +  move  to  next  bead  row)  x  #  bead  rows  + 
move  to  service  each  task  x  #  tasks  +  tour  closure  length 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  7/8)  29junl0  @  Baltimore,  ACC  14  /  36 

Analysis  of  the  single-sweep  tiling  algorithm 


TSP(n)  —  (bead  row  length  +  move  to  next  bead  row)  x  #  bead  rows  + 
move  to  service  each  task  x  #  tasks  +  tour  closure  length 


p- 


p+ 


®  For  a  Reeds-Shepp  car,  as  £  — ►  0: 

TSP(n)  <  (a/|Q[  +  $)h  +  ^  ( V\Q\  +  P 7r) 

<  16„I§1  +  +  (n  +  2  (V/|Q|  +  ■  w(f)  -a  £) 

•  t  =  (^)1/3  (i.e„  J|  =  2)  =>  TSP(n)  =  O(n^). 
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Analysis  of  the  single-sweep  tiling  algorithm 
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Analysis  of  the  single-sweep  tiling  algorithm 


»  For  a  Dubins  vehicle,  as  £  — >  0: 

TSP(n)  <  (y/\Q\  +  w(fi/2  +  ^  +  K^n  +  ^VTSl  +  K 

<  16p^-  +  y^|  Q|  +  16/c^P  +  ^n  +  rtn  +  2y^|  Q|  +  n 
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Analysis  of  the  single-sweep  tiling  algorithm 


V- 


P+ 


o  For  a  Dubins  vehicle,  as  £  — >  0: 

TSP(n )  <  (^\/\Q\  +  +  k)  +  (^  +  K)n  +  2 VT Q|  +  ^ 

<  16p^  +  vlQ|  T  16a:  +  in  +  aji  +  2y^|  Q\  +  a: 

o  The  a :n  term  grows  linearly  in  n  for  all  £  =>  TSP(n)  =  O(n) 
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The  recursive  sweep  tiling  algorithm 


®  Tile  Q  with  beads  such  that:  ^  ^  ~  n-1/3) 

®  Sweep  the  bead  rows,  visiting  one  target  per  non-empty  bead. 

®  Iterate,  using  at  the  /'-th  phase  a  "meta-bead”  composed  of  2,_1 
beads. 

®  After  log  n  phases,  visit  the  outstanding  targets  in  any  arbitrary  order, 
e.g.,  with  a  greedy  strategy. 
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The  recursive  sweep  tiling  algorithm 


®  Tile  Q  with  beads  such  that:  jgf  —  ^  (i.e.,  ^  ~  n-1/3) 

®  Sweep  the  bead  rows,  visiting  one  target  per  non-empty  bead. 

®  Iterate,  using  at  the  /'-th  phase  a  "meta-bead”  composed  of  2,_1 
beads. 

®  After  log  n  phases,  visit  the  outstanding  targets  in  any  arbitrary  order, 
e.g.,  with  a  greedy  strategy. 
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Analysis  of  the  recursive  algorithm 


a  Theorem:  For  a  Dubins  vehicle,  with  probability  one, 

'rr  £  24-J/pfeT  ^1  +  ln-^j 
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Summary  of  TSPs 


®  Lower  bound:  E[TSP(n)]  6  ft(n2/3) 

®  Upper  bound:  E[TSP(n)]  G  0(n2/3) 

®  TSP(n)  is  of  order  n2/3;  constant  factor  approximation  algorithms 
®  Computational  complexity  of  the  algorithms  is  of  order  n 
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Analysis  of  the  recursive  algorithm 


®  Theorem:  For  a  Dubins  vehicle,  with  probability  one, 

'SP(/ 

~n^ 


limsupT~T^  •  24 y/,| Q|  (l  -  T-  P 


3  vTSl, 


Outline  of  the  proof 


q  Pr(limn^00  #  tasks  remaining  after  phase  /*  >  24  log  n)  =  0 
a  Path  length  calculations: 

o  Phase  1  path  length  0  (^)  =  0  (n2/3)  (*.*  £  ~  n-1/3) 

o  Subsequent  phase  path  lengths  are  decreasing  geometric  series;  path 
length  for  all  /*  phases  is  0  (n2/3) 
o  Path  length  by  greedy  heuristic  is  O(logn) 
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Summary  of  TSPs 


®  Lower  bound:  E[TSP(n)]  G  fi(n2/3) 

®  Upper  bound:  E[TSP(n)]  G  0(n2/3) 

®  TSP(n)  is  of  order  n2/3;  constant  factor  approximation  algorithms 
®  Computational  complexity  of  the  algorithms  is  of  order  n 
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Outline  of  the  lecture 


Q  The  heavy  load  case 
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The  heavy  load  case:  nearest  neighbor  lower  bound 


®  Let  rin  be  the  number  of  outstanding  tasks  at  steady-state  under 
stable  policy  tt 

o  Calculate  (an  upper  bound  on)  expected  distance  from  an  arbitrary 
vehicle  configuration  to  closest  among  n n  points,  S*(n7r ) 

o  At  steady-state:  £  = 
o  Little’s  formula:  AT n  =  n n 
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The  multiple  sweep  tiling  algorithm 


The  single  vehicle  version 


Q  Tile  Q  with  beads  of  length 

i  =  c/A 

Q  Update  outstanding  task  list 

O  Execute  single  sweep  tiling 
algorithm 

O  Goto  2. 
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The  multiple  sweep  tiling  algorithm 


Analysis  of  the  multiple  sweep  algorithm 


The  single  vehicle  version 


O  Tile  Q  with  beads  of  length 

i  =  c/A 

Q  Update  outstanding  task  list 

O  Execute  single  sweep  tiling 
algorithm 

Q  Goto  2. 
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General  protocol 


o  Each  bead  can  be  treated  as  a  separate  queue,  with  Poisson  arrival 
process  with  intensity  A#  =  Ay^y 

o  The  vehicle  visits  each  bead  with  at  a  rate  no  smaller  than 
!±B  «  (single  sweep  path  length)-1 
o  The  system  time  is  no  greater  than  the  system  time  for  the 
corresponding  M/D/1  queue:  T*  <  -^  (l  +  \j^^) 
o  Optimize  over  £ 
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The  light  load  case 


The  light  load  case 


o  The  target  generation  rate  is  very  small:  X/m  — >  0+ 

In  such  case: 

o  Almost  surely  all  vehicles  will  have  enough  time  to  return  to  some 
’’loitering  station”  between  task  completion/generation  times 

o  The  problem  is  reduced  to  the  choice  of  the  loitering  stations  that 
minimizes  the  system  time 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT)  Dynamic  Vehicle  Routing  (Lecture  7/8)  29junl0  @  Baltimore,  ACC  24  /  36 


®  The  target  generation  rate  is  very  small:  X/m  — ►  0+ 

In  such  case: 

a  Almost  surely  all  vehicles  will  have  enough  time  to  return  to  some 
"loitering  station”  between  task  completion/generation  times 

®  The  problem  is  reduced  to  the  choice  of  the  loitering  stations  that 
minimizes  the  system  time 


Introducing  differential  constraints 


o  Novel  challenges: 

o  Vehicles  possibly  cannot  stop  (e.g.,  Dubins  vehicle,  Reeds-Shepp  car) 
o  Strategies  are  more  complex  than  defining  a  loitering  "point” 

o  How  many  of  the  results  from  the  Euclidean  case  carry  over  to  this 
case? 
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A  simple  lower  bound 


®  The  length  of  shortest  feasible  path  from  a  vehicle  positioned  at 
p  G  l2  to  an  arbitrary  point  q  e  Q  is  lower  bounded  by  ||q  —  p|| 

®  A  simple  lower  bound  on  T*  is  obtained  by  relaxing  differential 
constraints 

•  T*  >  HJQ) 

•  H-m(Q)  =  e  (^) 
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The  Median  Circling  (MC)  Policy 


Assign  "virtual”  generators  to  each  agent.  All  agents  do  the  following,  in 
parallel  (possibly  asynchronously): 

®  Update  the  generator  position  according  to  a  gradient  descent  law. 

®  Service  targets  in  own  region,  returning  to  a  "loitering  circle”  of 
radius  2.91p  centered  on  their  generator  position  when  done 
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The  Median  Circling  (MC)  Policy 


Assign  "virtual"  generators  to  each  agent.  All  agents  do  the  following,  in 
parallel  (possibly  asynchronously): 

a  Update  the  generator  position  according  to  a  gradient  descent  law. 

9  Service  targets  in  own  region,  returning  to  a  "loitering  circle"  of 
radius  2.91  p  centered  on  their  generator  position  when  done 
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Tighter  lower  bound  using  differential  constraints 


o  Consider  a  "frozen  moment  in  time" 
o  Consider  the  "modified  Voronoi"  diagram  of  the  vehicles, 
o  Relaxation:  approximate  vehicle  Voronoi  region  by  their  reachable  sets 
o  Optimize  over  the  vehicle  configurations 
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Illustration  of  the  MC  policy 


O 
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Tighter  lower  bound  using  differential  constraints 


o  Consider  a  "frozen  moment  in  time" 
o  Consider  the  "modified  Voronoi"  diagram  of  the  vehicles, 
o  Relaxation:  approximate  vehicle  Voronoi  region  by  their  reachable  sets 
o  Optimize  over  the  vehicle  configurations 
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The  Strip  Loitering  (SL)  policy 


Divide  the  environment  Q  into  strips  of  width  min 

Design  a  closed  loitering  path  that  bisects  the  strips.  All  vehicles  move  along 
this  path,  equally  spaced,  with  dynamic  regions  of  responsibility. 

Each  vehicle  services  targets  in  own  region,  returning  to  the  ’’nominal” 
position  on  the  loitering  path. 
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Illustration  of  the  SL  policy 


0  0 


0  0 


33333333 


00000000 


33333333 


00000000 
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Outline  of  the  lecture 


o  Phase  transition  in  the  light  load 
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Phase  transition  in  the  light  load 


Phase  transition  in  the  light  load 


®  We  have  two  policies:  Median  Circling  (MC),  and  Strip  Loitering 
(SL).  Which  is  better? 

2 

o  Define  the  non-holonomic  density  dp  — 

9  MC  is  optimal  when  dp  — >  0, 

9  SL  is  within  a  constant  factor  of  the  optimal  as  dp  — ►  +oo. 
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®  We  have  two  policies:  Median  Circling  (MC),  and  Strip  Loitering 
(SL).  Which  is  better? 

2 

o  Define  the  non-holonomic  density  dp  = 

9  MC  is  optimal  when  dp  — »  0, 

o  SL  is  within  a  constant  factor  of  the  optimal  as  dp  — >  +oo. 

o  phase  transition:  the  optimal  organization  changes  from  territorial 
(MC)  to  gregarious  (SL)  depending  on  the  non-holonomic  density  of 
the  agents. 
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Dynamic  Vehicle  Routing  Summary 


Euclidean 

vehicle 

Dubins  vehicle,  Reeds-Shepp  car 
Double  integrator,  Differential  drive 

E[TSP  Length] 

(n  — ►  oo) 

0(n5) 

Q(ni) 

T* 

®(5) 

T* 

igi  -  o) 

0 

3 

i 

roll-- 

0(m“5) 

T * 

(£  1ST  ^°°) 

0(m  2) 

Q(m~3) 
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Workshop  Structure  and  Schedule 


8:00-8:30am 

8:30-9:00am 

9:05-9:50am 

Coffee  Brea 
Lecture  #1 
Lecture  #2 

k 

Intro  to  dynamic  vehicle  routing 

Prelims:  graphs,  TSPs  and  queues 

9:55-10:40am 

Lecture  #3 

The  single-vehicle  DVR  problem 

10:40-ll:00am 

ll:00-ll:45pm 

ll:45-l:10pm 

l:10-2:10pm 

2:15-3:00pm 

3:00-3:20pm 

3:20-4:20pm 

Break 

Lecture  #4: 
Lunch  Break 
Lecture  #5: 
Lecture  #6: 
Coffee  Break 
Lecture  #7: 

The  multi-vehicle  DVR  problem 

Extensions  to  vehicle  networks 
Extensions  to  different  demand  models 

Extensions  to  different  vehicle  models 

4:25-4:40pm 

4:45-5:00pm 

Lecture  #8: 

Extensions  to  different  task  models 

Final  open-floor  discussion 
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Lecture  outline 


^  Models  of  vehicles  with  differential  constraints 
^  Traveling  salesperson  problems 
Q  The  heavy  load  case 
Q  The  light  load  case 
o  Phase  transition  in  the  light  load 
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Dynamic  Vehicle  Routing  for  Robotic  Networks 
Lecture  #8:  Different  Task  Models 
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Lecture  outline 


Q  Dynamic  Team  Forming 

Q  Three  Policies 

9  Complete  Team 
9  Task-Specific  Team  Policy 
9  Scheduled  Task-Specific  Team  Policy 


Q  Analysis  of  Policies 

9  Throughput  vs  System  Time 
9  Comparison  of  Results 
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Motivation  for  Team  Forming 


9  Group  of  vehicles  monitoring  a  region 
9  Several  different  sensing  modalities: 

©  electro-optical, 
o  infra-red, 

o  synthetic  aperture  radar, 

©  foliage  penetrating  radar, 

©  etc. 

9  Each  event  requires  a  subset  of  sensing  modalities 

9  Equip  each  vehicle  with  a  single  sensing  modality 
9  Form  appropriate  team  to  properly  assess  each  event 

How  do  we  create  teams  in  real-time  to  observe  each  event 

(service  each  request)? 
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Lecture  outline 


Q  Dynamic  Team  Forming 
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Load  Factor  and  Stability 


»  Ra  G  {0,  l}k  is  zero-one  column  vector  recording  services  required 
task-type  a. 

o  on-site  service  for  task-type  a  is  sa 
9  rrij  vehicles  provide  service  rj. 

Necessary  stability  condition: 


[Ri  •••  Ric] 


Aisi 

mi 

< 

_^kSk:_ 

mk_ 

Load  factor  is  now  a  vector 
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Dynamic  Team  Forming 


Set  of  services  {ri, . . . ,  rk} . 

Vehicle  properties: 

9  k  different  vehicle  types. 

9  Vehicle  type  j  6.  {1, ,  k},  can  provide  only  service  rj. 

Task  (demand)  model: 

9  Poisson  and  Uniform  arrivals 

®  Each  task  requires  a  subset  of  services  in  (ri, . . . ,  rk}. 

9  K.  different  types  of  tasks 
®  Tasks  of  type  a.  arrive  at  rate  Aa 

®  Task  completed  when  required  vehicles  simultaneously  spend  on-site 
service  time  at  location. 

S.  L.  Smith  and  F.  Bullo.  The  dynamic  team  forming  problem:  Throughput  and  delay  for  unbiased 
policies.  Systems  &  Control  Letters,  58(10-11):709— 715,  2009 
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Example  of  Team  Forming 


®  k  =  4  different  services,  {ri,  /-2,  r3,  r 4}. 

9m  —  8  vehicles,  two  of  each  type:  mj  —  2  for  j  G  (1, 2, 3, 4}. 
9  K,  =  6  task  types,  {ri},  {r2},  {r3},  {r4},  (ri,  r3),  {r2,  r4}. 

{ri,r3} 

© 


Task  type  a  =  {ri,  73}  has  on-site  service  sa,  arrival  rate  Aa,  and 

Ra  =  [1  0  1  0]T. 
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Lecture  outline 


^  Three  Policies 

®  Complete  Team 
®  Task-Specific  Team  Policy 
®  Scheduled  Task-Specific  Team  Policy 
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Policy  1:  Complete  Team  Policy 


Complete  Team  Policy 

1:  Form  min{AT7i, .  . .  m^}  teams  of  k  vehicles,  each  team 
contains  one  vehicle  of  each  type. 

2:  Have  each  team  meet  and  move  as  a  single  entity. 

3:  In  each  region  run  UTSP  policy  (from  Lecture  3) . 


Can  also  use  Divide  &  Conquer  policy  for  each  team 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT) 


Dynamic  Vehicle  Routing  (Lecture  8/8)  29junl0  ©  Baltimore,  ACC 


11  /  24 


Task-Type  Unbiased  Policies 


For  a  policy  7 r: 

®  System  time  of  each  task-type  Tnl, . . . , 

®  Feasible  set  of  system  times  are  subset  of  IR^ 

®  Optimization  space  similar  to  priority  queues,  but  with  teaming 

To  simplify,  consider  task-type  unbiased  policies 

=  T 7^2  =  •  •  •  =  TffX  =  .  T IX 

and  the  optimization:  inf*-  T n. 
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Policy  1:  Complete  Team  Policy 


•  ^ 

© 

>  © 

©  Q 


•  M 


©  O 


V 


o  © 

#  V 


*  Two  services  y,  b 
a  3  task- types  y,  b,  {y,  b}. 
o  4  vehicles 
o  2  yellow 
a  2  blue 
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Policy  2:  Task-specific  Team  Policy 


o  rrij  vehicles  provide  service  rj. 
a  rj  appears  in  ej [Ri  •  •  •  Rjc]1jc  task  types. 

«  If  rrij  >  ej  [Ri  •  •  •  Rjc] ljc  enough  vehicles  of  type  j  to  create 
dedicated  team  for  each  task  type. 

9  Create  mTST  teams,  where: 


mTST  : — 


min 

j 


mj  \ 

eJTRljcj_ 


Task-Specific  Team  Policy 

1:  For  each  of  the  /C  task  types,  create  mTST  teams  of 
vehicles . 

2:  Service  each  task  by  one  of  its  mTST  corresponding 
teams,  according  to  the  UTSP  policy. 
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Policy  3:  Preliminary  Result 


Definition  (Service  schedule) 


A  partition  of  task  types  into  L  time  slots,  such  that: 
o  each  type  appears  in  exactly  one  time  slot,  and 
o  task  types  in  each  time  slot  are  pairwise  disjoint. 


{1} 

{2} 

{3} 

{4} 

{5} 

{1,2} 

{3,4} 

{1,4} 

{2,5} 

{1,2,3} 

{1,3,5} 

{1,4,6} 

{6} 

{5,6} 

{3,6} 

{4,5,6} 

{2,4,6} 

{2, 3, 5} 

0  t-Q  3£g  4  t-Q  5£g  6£g  time 
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Policy  2:  Task-Specific  Team  Policy 


~ir~© 

« 

Q 

• 

• 

9 

1 

3  © 

A 

• 

• 

© 

• 

• 

© 

•  ^ 

© 

c 

> 

•  ©  • 

•  © 

► 

o  task  types: 

M>{&}>{y> b } 

o  two  vehicles  of  each  type 

o  y,  b  each  appear  in  two 
task-types 


mTST  —  1 
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Policy  3:  Scheduled  Task-Specific  Team  Policy 


Scheduled  Task-specific  team  policy 

Partition  Q  into  min ;{m;}  regions  and  assign  one  robot 
of  each  type  to  each  region. 

1:  In  each  region  form  a  queue  for  each  task  type. 

2:  For  each  time  slot  in  the  schedule: 

Q  Divide  robots  into  teams  to  form  required  task 
types . 

Q  For  each  team,  service  corresponding  queue  with 
TSP  tour. 
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Policy  3:  Scheduled  Task-Specific  Team  Policy 


Service  schedule: 

9  two  time  slots  L  —  2 
®  slot  one:  {y},  {b} 

9  slot  two:  {y,  b} 
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Assumptions  for  Analysis 


Assumptions: 

Q  m;  —  m/k  for  each  vehicle  type  /. 

Q  Xa  =  A//C  for  each  task-type  cr. 

O  on-site  service  has  mean  s  and  is  upper  bounded  by  smax. 

O  pJC  of  the  K.  task-types  require  service  rj,  where  p  6  [l//r,  1]. 


9  With  assumptions,  necessary  stability  condition  becomes 

A  J_ 

m  pks 

•  Define  per-vehicle  throughput  as  Bm  A/m. 


FB,  EF,  MP,  KS,  SLS  (UCSB,  MIT) 


Dynamic  Vehicle  Routing  (Lecture  8/8)  29junl0  ©  Baltimore,  ACC 


19  /  24 


Lecture  outline 


Q  Analysis  of  Policies 

o  Throughput  vs  System  Time 
o  Comparison  of  Results 
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Throughput  vs  System  Time  Profile 
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Lecture  outline 


Q  Dynamic  Team  Forming 

Q  Three  Policies 

o  Complete  Team 
o  Task-Specific  Team  Policy 
o  Scheduled  Task-Specific  Team  Policy 


Q  Analysis  of  Policies 

o  Throughput  vs  System  Time 
o  Comparison  of  Results 
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System  Time  for  each  Policy 


T  min 

7”ord 

BCr\t 

Lower  bound  (7"*) 

\fk 

k 

1 

pks 

Complete  Team 

\fk 

k 

x 

ks 

Task-Specific 

VpkK 

pk)C 

1 

pks 

Scheduled  Task-Specific 

LVk 

Lk 

JC 

Smax  Lk 

where  L  £  [p/C,  K] 

Best  policies  for  different  scenarios: 

o  If  throughput  is  low,  then  use  complete  team 
a  If  p  is  close  to  1,  then  use  complete  team 
o  If  p  is  close  to  l//c,  then  for  best  capacity  use 
o  Task-Specific  if  enough  vehicles 
9  Scheduled  Task-Specific  otherwise 
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Workshop  Structure  and  Schedule 


8:00-8:30am 

Coffee  Break 

8:30-9:00am 

Lecture  #1: 

9:05-9:50am 

Lecture  #2: 

9:55-10:40am 

Lecture  #3: 

10:40-ll:00am 

Break 

ll:00-ll:45pm 

Lecture  #4: 

ll:45-l:10pm 

Lunch  Break 

l:10-2:10pm 

Lecture  #5: 

2:15-3:00pm 

Lecture  #6: 

3:00-3:20pm 

Coffee  Break 

3:20-4:20pm 

Lecture  #7: 

4:25-4:40pm 

4:45-5:00pm 

Lecture  #8: 

Intro  to  dynamic  vehicle  routing 
Prelims:  graphs,  TSPs  and  queues 
The  single-vehicle  DVR  problem 

The  multi-vehicle  DVR  problem 

Extensions  to  vehicle  networks 
Extensions  to  different  demand  models 

Extensions  to  different  vehicle  models 
Extensions  to  different  task  models 
Final  open-floor  discussion 
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